Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-08 Thread Amit Kapila
On Tue, Jun 9, 2015 at 10:56 AM, Fujii Masao wrote: > > On Tue, Jun 9, 2015 at 1:04 PM, Amit Kapila wrote: > > On Tue, Jun 9, 2015 at 9:09 AM, Fujii Masao wrote: > >> > >> On Tue, May 12, 2015 at 10:42 PM, Andrew Dunstan > >> wrote: > >> > Map basebackup tablespaces using a tablespace_map file

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-06-08 Thread Magnus Hagander
On Jun 9, 2015 6:00 AM, "Michael Paquier" wrote: > > Hi all, > > I should have noticed that before, but it happens that pg_stat_ssl > leaks information about the SSL status of all the users connected to a > server. Let's imagine for example: > 1) Session 1 connected through SSL with a superuser: >

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-08 Thread Fujii Masao
On Tue, Jun 9, 2015 at 1:04 PM, Amit Kapila wrote: > On Tue, Jun 9, 2015 at 9:09 AM, Fujii Masao wrote: >> >> On Tue, May 12, 2015 at 10:42 PM, Andrew Dunstan >> wrote: >> > Map basebackup tablespaces using a tablespace_map file >> > >> > Windows can't reliably restore symbolic links from a tar

[HACKERS] Useless mention of RMGRDESCSOURCES in src/bin/pg_rewind/Makefile

2015-06-08 Thread Michael Paquier
Hi all, pg_rewind's Makefile uses RMGRDESCSOURCES: EXTRA_CLEAN = $(RMGRDESCSOURCES) xlogreader.c However this variable is defined only in the Makefile of pg_xlogdump so it has no utility in this case. Attached is a cleanup patch. Regards, -- Michael diff --git a/src/bin/pg_rewind/Makefile b/src/b

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-08 Thread Amit Kapila
On Tue, Jun 9, 2015 at 9:09 AM, Fujii Masao wrote: > > On Tue, May 12, 2015 at 10:42 PM, Andrew Dunstan wrote: > > Map basebackup tablespaces using a tablespace_map file > > > > Windows can't reliably restore symbolic links from a tar format, so > > instead during backup start we create a tablesp

[HACKERS] Information of pg_stat_ssl visible to all users

2015-06-08 Thread Michael Paquier
Hi all, I should have noticed that before, but it happens that pg_stat_ssl leaks information about the SSL status of all the users connected to a server. Let's imagine for example: 1) Session 1 connected through SSL with a superuser: =# create role toto login; CREATE ROLE =# select * from pg_stat_

[HACKERS] Aggregate Supporting Functions

2015-06-08 Thread David Rowley
I believe this is an idea that's been discussed before, but I'm not exactly sure where that happened: Overview: The idea is that we skip a major chunk of processing in situations like: SELECT avg(x),sum(x),count(x) FROM bigtable; Because avg(x) already technically knows what the values of sum(x

Re: [HACKERS] Memory leak with XLogFileCopy since de768844 (WAL file with .partial)

2015-06-08 Thread Fujii Masao
On Tue, Jun 9, 2015 at 6:25 AM, Heikki Linnakangas wrote: > On 06/08/2015 09:04 PM, Fujii Masao wrote: >> >> On Mon, Jun 8, 2015 at 11:52 AM, Michael Paquier >> wrote: >>> >>> On Fri, Jun 5, 2015 at 10:45 PM, Fujii Masao wrote: Why don't we call InstallXLogFileSegment() at the end of XL

[HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-08 Thread Fujii Masao
On Tue, May 12, 2015 at 10:42 PM, Andrew Dunstan wrote: > Map basebackup tablespaces using a tablespace_map file > > Windows can't reliably restore symbolic links from a tar format, so > instead during backup start we create a tablespace_map file, which is > used by the restoring postgres to creat

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-08 Thread Amit Kapila
On Tue, Jun 9, 2015 at 12:27 AM, Andrew Dunstan wrote: > > On 06/08/2015 11:16 AM, Amit Kapila wrote: >> >> >> I have to retry that operation, but for me unlink hasn't deleted >> the file on Windows, may be I am not doing properly, but in >> anycase why we want to throw error for such a case, why

Re: [HACKERS] pg_stat_archiver issue with aborted archiver

2015-06-08 Thread Michael Paquier
On Tue, Jun 9, 2015 at 4:23 AM, Fujii Masao wrote: > On Mon, Jun 8, 2015 at 5:17 PM, Julien Rouhaud > wrote: >> Le 08/06/2015 05:56, Michael Paquier a écrit : >>> On Sun, Jun 7, 2015 at 1:11 AM, Julien Rouhaud >>> wrote: I just noticed that if the archiver aborts (for instance if the a

Re: [HACKERS] Memory leak with XLogFileCopy since de768844 (WAL file with .partial)

2015-06-08 Thread Michael Paquier
On Tue, Jun 9, 2015 at 6:25 AM, Heikki Linnakangas wrote: > I'm still not sure if I should've just reverted that refactoring, to make > XLogFileCopy() look the same in master and back-branches, which makes > back-patching easier, or keep the refactoring, because it makes the code > slightly nicer.

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread David Gould
On Mon, 8 Jun 2015 13:03:56 -0300 Claudio Freire wrote: > > Ohmygosh, you have to rpm install a bunch of -devel stuff? What a massive > > hardship. > > It's not about the 5 minutes of compile time, it's about the signalling. > > Just *when* is git ready for testing? You don't know from the outs

Re: [HACKERS] Cancel race condition

2015-06-08 Thread Tom Lane
Shay Rojansky writes: > My questions/comments: >- Does PostgreSQL *guarantee* that once the connection used to send the >cancellation request is closed by the server, the cancellation has been >delivered (as mentioned by Tom)? In other words, should I be designing a >.NET driver ar

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread Gavin Flower
On 09/06/15 00:59, David Gould wrote: I think Alphas are valuable and useful and even more so if they have release notes. For example, some of my clients are capable of fetching sources and building from scratch and filing bug reports and are often interested in particular new features. They even

Re: [HACKERS] Memory leak with XLogFileCopy since de768844 (WAL file with .partial)

2015-06-08 Thread Heikki Linnakangas
On 06/08/2015 09:04 PM, Fujii Masao wrote: On Mon, Jun 8, 2015 at 11:52 AM, Michael Paquier wrote: On Fri, Jun 5, 2015 at 10:45 PM, Fujii Masao wrote: Why don't we call InstallXLogFileSegment() at the end of XLogFileCopy()? If we do that, the risk of memory leak you're worried will disappear a

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Alvaro Herrera
Josh Berkus wrote: > On 06/08/2015 12:48 PM, Alvaro Herrera wrote: > > Well, reputation-wise we're already losing every time somebody's server > > crashes on 9.4.2 and finds it won't start, where it did start fine with > > 9.4.1. Maybe they simply wanted to change shared_buffers and the server > >

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Josh Berkus
On 06/08/2015 12:48 PM, Alvaro Herrera wrote: > Well, reputation-wise we're already losing every time somebody's server > crashes on 9.4.2 and finds it won't start, where it did start fine with > 9.4.1. Maybe they simply wanted to change shared_buffers and the server > won't start anymore. Some p

Re: [HACKERS] pg_stat_*_columns?

2015-06-08 Thread Joel Jacobson
So I've heard from Magnus Hagander today IRL at our Stockholm PostgreSQL User Group meeting where we discussed this idea. He told me the overhead in the statistics collector is mainly when reading from it, not that much when writing to it. Magnus idea was to first optimize the collector to make it

[HACKERS] Cancel race condition

2015-06-08 Thread Shay Rojansky
Hi everyone. I'm working on Npgsql and have run into a race condition when cancelling. The issue is described in the following 10-year-old thread, and I'd like to make sure things are still the same: http://www.postgresql.org/message-id/27126.1126649...@sss.pgh.pa.us My questions/comments: -

Re: [HACKERS] could not truncate directory "pg_subtrans": apparent wraparound

2015-06-08 Thread Dan Langille
If there's anything I can try on my servers to help diagnose the issues, please let me know. If desired, I can arrange access for debugging. On Sat, Jun 6, 2015 at 12:51 AM, Thomas Munro wrote: > On Sat, Jun 6, 2015 at 1:25 PM, Alvaro Herrera > wrote: > > Thomas Munro wrote: > > > >> My idea w

Re: [HACKERS] reaper should restart archiver even on standby

2015-06-08 Thread Alvaro Herrera
Fujii Masao wrote: > Hi, > > When the archiver exits, currently reaper() restarts it only while > the postmaster state is PM_RUN. This is OK in 9.4 or before because > the archiver could be running on that state. But in 9.5, we can set > archive_mode to "always" and start the archiver even on the

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Joshua D. Drake
On 06/08/2015 12:48 PM, Alvaro Herrera wrote: Well, reputation-wise we're already losing every time somebody's server crashes on 9.4.2 and finds it won't start, where it did start fine with 9.4.1. Maybe they simply wanted to change shared_buffers and the server won't start anymore. Some peopl

[HACKERS] reaper should restart archiver even on standby

2015-06-08 Thread Fujii Masao
Hi, When the archiver exits, currently reaper() restarts it only while the postmaster state is PM_RUN. This is OK in 9.4 or before because the archiver could be running on that state. But in 9.5, we can set archive_mode to "always" and start the archiver even on the standby. So I think that reaper

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Alvaro Herrera
Joshua D. Drake wrote: > > On 06/08/2015 12:31 PM, Alvaro Herrera wrote: > > > >Joshua D. Drake wrote: > > > >>If we release on Friday that is the 12th, PgCon is starts the 16th and there > >>is a weekend in between. If there is an unknown regression or new bug that > >>is severe, are we going to

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Joshua D. Drake
On 06/08/2015 12:31 PM, Alvaro Herrera wrote: Joshua D. Drake wrote: If we release on Friday that is the 12th, PgCon is starts the 16th and there is a weekend in between. If there is an unknown regression or new bug that is severe, are we going to have the resources to resolve it? ISTM if t

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Alvaro Herrera
Joshua D. Drake wrote: > If we release on Friday that is the 12th, PgCon is starts the 16th and there > is a weekend in between. If there is an unknown regression or new bug that > is severe, are we going to have the resources to resolve it? ISTM if that happens, we're no worse off than currently

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Joshua D. Drake
On 06/08/2015 12:21 PM, Tom Lane wrote: Andres Freund writes: On 2015-06-08 14:18:22 -0400, Tom Lane wrote: As I saw it, on Friday it was not clear whether we would be able to do a release this week. Now it's Monday, and we still have a rather long list of issues Well, these issues aren't

Re: [HACKERS] last_analyze/last_vacuum not being updated

2015-06-08 Thread Tom Lane
Peter Eisentraut writes: > I'm looking at a case on 9.4.1 where the last_analyze and last_vacuum > stats for a handful of tables seem stuck. They don't update after > running an ANALYZE or VACUUM command, and they don't react to > pg_stat_reset_single_table_counters(). All of the affected tables

Re: [HACKERS] last_analyze/last_vacuum not being updated

2015-06-08 Thread Alvaro Herrera
Peter Eisentraut wrote: > I'm looking at a case on 9.4.1 where the last_analyze and last_vacuum > stats for a handful of tables seem stuck. They don't update after > running an ANALYZE or VACUUM command, and they don't react to > pg_stat_reset_single_table_counters(). All of the affected tables a

Re: [HACKERS] pg_stat_archiver issue with aborted archiver

2015-06-08 Thread Fujii Masao
On Mon, Jun 8, 2015 at 5:17 PM, Julien Rouhaud wrote: > Le 08/06/2015 05:56, Michael Paquier a écrit : >> On Sun, Jun 7, 2015 at 1:11 AM, Julien Rouhaud >> wrote: >>> I just noticed that if the archiver aborts (for instance if the >>> archive_command exited with a return code > 127), >>> pg_stat_

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Tom Lane
Andres Freund writes: > On 2015-06-08 14:18:22 -0400, Tom Lane wrote: >> As I saw it, on Friday it was not clear whether we would be able to do a >> release this week. Now it's Monday, and we still have a rather long list >> of issues > Well, these issues aren't regressions, they're "just" gener

[HACKERS] last_analyze/last_vacuum not being updated

2015-06-08 Thread Peter Eisentraut
I'm looking at a case on 9.4.1 where the last_analyze and last_vacuum stats for a handful of tables seem stuck. They don't update after running an ANALYZE or VACUUM command, and they don't react to pg_stat_reset_single_table_counters(). All of the affected tables are system catalogs, some shared,

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Robert Haas
On Mon, Jun 8, 2015 at 2:18 PM, Tom Lane wrote: > As I saw it, on Friday it was not clear whether we would be able to do a > release this week. Now it's Monday, and we still have a rather long list > of issues, and apparently Andres isn't all that happy even with the fixes > that have gone in, be

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-08 Thread Andrew Dunstan
On 06/08/2015 11:16 AM, Amit Kapila wrote: On Mon, Jun 8, 2015 at 6:39 PM, Andrew Dunstan > wrote: On 06/08/2015 12:08 AM, Amit Kapila wrote: How about if it is just a flat file with same name as tablespace link, why we want to give erro

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Andres Freund
On 2015-06-08 14:18:22 -0400, Tom Lane wrote: > As I saw it, on Friday it was not clear whether we would be able to do a > release this week. Now it's Monday, and we still have a rather long list > of issues Well, these issues aren't regressions, they're "just" general problems we need to fix. An

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Tom Lane
Robert Haas writes: > It's not exactly going into a black hole, but there was some > communication between Tom and Andres on Friday that left Andres with > the impression that if he spent the weekend testing the new code for > problems and things went well, we'd be able to get a release this > wee

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Andres Freund
On 2015-06-08 13:16:00 -0400, Robert Haas wrote: > It's not exactly going into a black hole, but there was some > communication between Tom and Andres on Friday that left Andres with > the impression that if he spent the weekend testing the new code for > problems and things went well, we'd be able

Re: [HACKERS] Memory leak with XLogFileCopy since de768844 (WAL file with .partial)

2015-06-08 Thread Fujii Masao
On Mon, Jun 8, 2015 at 11:52 AM, Michael Paquier wrote: > On Fri, Jun 5, 2015 at 10:45 PM, Fujii Masao wrote: >> Why don't we call InstallXLogFileSegment() at the end of XLogFileCopy()? >> If we do that, the risk of memory leak you're worried will disappear at all. > > Yes, that looks fine, XLogFi

Re: [HACKERS] Dependency between bgw_notify_pid and bgw_flags

2015-06-08 Thread Robert Haas
On Mon, Jun 8, 2015 at 1:51 PM, Alvaro Herrera wrote: > Robert Haas wrote: > >> After studying this, I think it's a bug. See this comment: >> >> * Normal child backends can only be launched when we are in PM_RUN or >> * PM_HOT_STANDBY state. (We also allow launch of normal >> * child backends

Re: [HACKERS] Restore-reliability mode

2015-06-08 Thread Bruce Momjian
On Mon, Jun 8, 2015 at 07:48:36PM +0200, Andres Freund wrote: > On 2015-06-08 13:44:05 -0400, Bruce Momjian wrote: > > I understand the overreaction/underreaction debate. Here were my goals > > in this discussion: > > > > 1. stop worry about the 9.5 timeline so we could honestly assess our > >

Re: [HACKERS] Dependency between bgw_notify_pid and bgw_flags

2015-06-08 Thread Alvaro Herrera
Robert Haas wrote: > After studying this, I think it's a bug. See this comment: > > * Normal child backends can only be launched when we are in PM_RUN or > * PM_HOT_STANDBY state. (We also allow launch of normal > * child backends in PM_WAIT_BACKUP state, but only for superusers.) > * In ot

Re: [HACKERS] Restore-reliability mode

2015-06-08 Thread Andres Freund
On 2015-06-08 13:44:05 -0400, Bruce Momjian wrote: > I understand the overreaction/underreaction debate. Here were my goals > in this discussion: > > 1. stop worry about the 9.5 timeline so we could honestly assess our > software - *done* > 2. seriously address multi-xact issues without 9.5

Re: [HACKERS] Restore-reliability mode

2015-06-08 Thread Bruce Momjian
On Sat, Jun 6, 2015 at 03:58:05PM -0400, Noah Misch wrote: > On Fri, Jun 05, 2015 at 08:25:34AM +0100, Simon Riggs wrote: > > This whole idea of "feature development" vs reliability is bogus. It > > implies people that work on features don't care about reliability. Given > > the fact that many of

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-08 Thread Alvaro Herrera
Robert Haas wrote: > Why not? I think that if we encounter some sort of situation that we > think should never happen, throwing an error is exactly what we > *should* do. Particularly when it comes to things like removing > files, it is very dangerous for the database to proceed if the > situati

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-08 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Jun 8, 2015 at 1:23 PM, Alvaro Herrera > wrote: > > (My personal alarm bells go off when I see autovac_naptime=15min or > > more, but apparently not everybody sees things that way.) > > Uh, I'd echo that sentiment if you did s/15min/1min/ Yeah, well, that too I gue

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-08 Thread Robert Haas
On Mon, Jun 8, 2015 at 1:23 PM, Alvaro Herrera wrote: > Andres Freund wrote: >> On June 8, 2015 7:06:31 PM GMT+02:00, Alvaro Herrera >> wrote: >> >I might be misreading the code, but PMSIGNAL_START_AUTOVAC_LAUNCHER >> >only causes things to happen (i.e. a new worker to be started) when >> >autov

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-08 Thread Andres Freund
On 2015-06-08 14:23:32 -0300, Alvaro Herrera wrote: > Sure. I just concern that we might be putting excessive trust on > emergency workers being launched at a high pace. I'm not sure what to do about that. I mean, it'd not be hard to simply ignore naptime upon wraparound, but I'm not sure that'd

Re: [HACKERS] Dependency between bgw_notify_pid and bgw_flags

2015-06-08 Thread Robert Haas
On Thu, Jun 4, 2015 at 8:52 AM, Ashutosh Bapat wrote: > Documentation here http://www.postgresql.org/docs/devel/static/bgworker.html > does not indicate any relation between the fields bgw_notify_pid and > bgw_flags of BackgroundWorker structure. But in one has to set > BGWORKER_BACKEND_DATABASE_C

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-08 Thread Alvaro Herrera
Andres Freund wrote: > On June 8, 2015 7:06:31 PM GMT+02:00, Alvaro Herrera > wrote: > >I might be misreading the code, but PMSIGNAL_START_AUTOVAC_LAUNCHER > >only causes things to happen (i.e. a new worker to be started) when > >autovacuum is disabled. If autovacuum is enabled, postmaster > >r

Re: [HACKERS] [idea] more aggressive join pushdown on postgres_fdw

2015-06-08 Thread Robert Haas
On Fri, Jun 5, 2015 at 5:51 AM, Shigeru HANADA wrote: > 2015/06/05 6:43、Robert Haas のメール: >> On Sat, May 30, 2015 at 9:03 PM, Kouhei Kaigai wrote: >> Neat idea. This ties into something I've thought about and mentioned >> before: what if the innerrel is local, but there's a replicated copy >> o

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Alvaro Herrera
Bruce Momjian wrote: > On Mon, Jun 8, 2015 at 02:01:52PM -0300, Alvaro Herrera wrote: > > > OK, are these fixed in 9.4.2 or would the same failure happen in 9.4.3? > > > > The fixes are not yet in any released branch, hence the rush to get > > these out. > > OK, now I understand. :-O We have

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Robert Haas
On Mon, Jun 8, 2015 at 1:08 PM, Bruce Momjian wrote: > OK, now I understand. :-O We have known failures that are not patched, > hence the desire for a release. > > I am a little concerned we are getting into a case where community > members dedicated to this issue are asking for a release, and i

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-08 Thread Andres Freund
On June 8, 2015 7:06:31 PM GMT+02:00, Alvaro Herrera wrote: >Andres Freund wrote: > >> A first version to address this problem can be found appended to this >> email. >> >> Basically it does: >> * Whenever more than MULTIXACT_MEMBER_SAFE_THRESHOLD are used, signal >> autovacuum once per member

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Bruce Momjian
On Mon, Jun 8, 2015 at 02:01:52PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Mon, Jun 8, 2015 at 12:39:24PM -0400, Robert Haas wrote: > > > On Mon, Jun 8, 2015 at 12:36 PM, Bruce Momjian wrote: > > > > Yeah, I think if we needed this out in an emergency, we would do it, but > > >

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread Magnus Hagander
On Mon, Jun 8, 2015 at 7:01 PM, Alvaro Herrera wrote: > David G. Johnston wrote: > > On Mon, Jun 8, 2015 at 12:14 PM, Geoff Winkless > wrote: > > > > ​I can see that, and can absolutely get behind the idea of a nightly > being > > > flagged as an alpha, since it should involve next to no develop

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-08 Thread Alvaro Herrera
Andres Freund wrote: > A first version to address this problem can be found appended to this > email. > > Basically it does: > * Whenever more than MULTIXACT_MEMBER_SAFE_THRESHOLD are used, signal > autovacuum once per members segment > * For both members and offsets, once hitting the hard limi

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread Bruce Momjian
On Mon, Jun 8, 2015 at 12:32:45PM -0400, David G. Johnston wrote: > On Mon, Jun 8, 2015 at 12:14 PM, Geoff Winkless wrote: > > On 8 June 2015 at 17:03, Claudio Freire wrote: > > It's not about the 5 minutes of compile time, it's about the > signalling. > > Just *wh

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread Alvaro Herrera
David G. Johnston wrote: > On Mon, Jun 8, 2015 at 12:14 PM, Geoff Winkless wrote: > > ​I can see that, and can absolutely get behind the idea of a nightly being > > flagged as an alpha, since it should involve next to no developer time. > > > ​Nightly where? This is an international community.

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Alvaro Herrera
Bruce Momjian wrote: > On Mon, Jun 8, 2015 at 12:39:24PM -0400, Robert Haas wrote: > > On Mon, Jun 8, 2015 at 12:36 PM, Bruce Momjian wrote: > > > Yeah, I think if we needed this out in an emergency, we would do it, but > > > based on the volume of recent releases, it would be hard. Are we seein

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread Stephen Frost
David, * David G. Johnston (david.g.johns...@gmail.com) wrote: > On Mon, Jun 8, 2015 at 12:03 PM, Claudio Freire > wrote: > > Just *when* is git ready for testing? You don't know from the outside. > > > > I do lurk here a lot and still am unsure quite often. > > > > Even simply releasing an alpha

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Bruce Momjian
On Mon, Jun 8, 2015 at 01:53:42PM -0300, Alvaro Herrera wrote: > * people with the wrong oldestMulti setting in pg_control (which would > be due to a buggy pg_upgrade being used long ago) will be unable to > start if they upgrade to 9.3.7 or 9.3.8. A solution for them would be > to downgrade to 9

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Bruce Momjian
On Mon, Jun 8, 2015 at 12:39:24PM -0400, Robert Haas wrote: > On Mon, Jun 8, 2015 at 12:36 PM, Bruce Momjian wrote: > > Yeah, I think if we needed this out in an emergency, we would do it, but > > based on the volume of recent releases, it would be hard. Are we seeing > > user reports of failure

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Alvaro Herrera
Bruce Momjian wrote: > On Mon, Jun 8, 2015 at 11:40:43AM -0400, Tom Lane wrote: > > Robert Haas writes: > > > So, when shall we do all of this releasing? It seems like we could do > > > stage-one of the multixact fixing this week, and then figure out how > > > to do the other stuff after PGCon.

Re: [HACKERS] pg_stat_*_columns?

2015-06-08 Thread Robert Haas
On Fri, Jun 5, 2015 at 7:51 AM, Joel Jacobson wrote: > Would others find it useful to see per column statistics about accesses to > specific columns? A probably serious and maybe not entirely obvious problem with this is that it would increase the amount of statistical information we keep by a pr

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread Andres Freund
On 2015-06-08 12:16:34 -0400, David G. Johnston wrote: > ​IIUC the master branch is always ready for testing. I don't really think so. For one we often find bugs ourselves quite quickly. But more importantly, I'd much rather have users use their precious (and thus limited!) time to test when the

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Robert Haas
On Mon, Jun 8, 2015 at 12:36 PM, Bruce Momjian wrote: > Yeah, I think if we needed this out in an emergency, we would do it, but > based on the volume of recent releases, it would be hard. Are we seeing > user reports of failures even on the newest released versions, or are > these preventive fix

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-08 Thread Andres Freund
On 2015-06-08 15:15:04 +0200, Andres Freund wrote: > 1) the autovacuum trigger logic isn't perfect yet. I.e. especially with > autovacuum=off you can get into situations where emergency vacuums > aren't started when necessary. This is particularly likely to happen > if either very large multi

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Bruce Momjian
On Mon, Jun 8, 2015 at 11:40:43AM -0400, Tom Lane wrote: > Robert Haas writes: > > So, when shall we do all of this releasing? It seems like we could do > > stage-one of the multixact fixing this week, and then figure out how > > to do the other stuff after PGCon. Alternatively, we can let the

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread David G. Johnston
On Mon, Jun 8, 2015 at 12:14 PM, Geoff Winkless wrote: > On 8 June 2015 at 17:03, Claudio Freire wrote: > >> It's not about the 5 minutes of compile time, it's about the signalling. >> >> Just *when* is git ready for testing? You don't know from the outside. >> >> I do lurk here a lot and still

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread David G. Johnston
On Mon, Jun 8, 2015 at 12:03 PM, Claudio Freire wrote: > Just *when* is git ready for testing? You don't know from the outside. > > I do lurk here a lot and still am unsure quite often. > > Even simply releasing an alpha *tarball* would be useful enough. What > is needed is the signal to test, ra

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread Geoff Winkless
On 8 June 2015 at 17:03, Claudio Freire wrote: > It's not about the 5 minutes of compile time, it's about the signalling. > > Just *when* is git ready for testing? You don't know from the outside. > > I do lurk here a lot and still am unsure quite often. > > Even simply releasing an alpha *tarbal

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-08 Thread Robert Haas
On Mon, Jun 8, 2015 at 11:16 AM, Amit Kapila wrote: > I have to retry that operation, but for me unlink hasn't deleted > the file on Windows, may be I am not doing properly, but in > anycase why we want to throw error for such a case, why > can't we just ignore and create a symlink with the same n

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread Claudio Freire
On Mon, Jun 8, 2015 at 12:22 PM, Geoff Winkless wrote: > On 8 June 2015 at 16:01, Robert Haas wrote: >> >> On Mon, Jun 8, 2015 at 9:21 AM, Geoff Winkless >> wrote: >> > Wow! I never knew there were all these people out there who would be >> > rushing >> > to help test if only the PG developers r

Re: [HACKERS] Initializing initFileRelationIds list via write is unsafe

2015-06-08 Thread Robert Haas
On Sat, Jun 6, 2015 at 11:51 PM, Tom Lane wrote: > This suggests that CLOBBER_CACHE_ALWAYS is actually missing a pretty > large part of the cache behavioral space. Maybe we should devise some > sort of "CLOBBER_CACHE_RANDOMLY" option that would inject cache flush > events more selectively, perhap

Re: [HACKERS] CREATE POLICY and RETURNING

2015-06-08 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: > On 6 June 2015 at 22:16, Peter Geoghegan wrote: > > On Fri, Oct 17, 2014 at 5:34 AM, Andres Freund > > wrote: > >> On 2014-10-17 14:57:03 +0800, Craig Ringer wrote: > >>> On 10/17/2014 02:49 AM, Robert Haas wrote: > >>> > I think you could proba

Re: [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Tom Lane
Robert Haas writes: > So, when shall we do all of this releasing? It seems like we could do > stage-one of the multixact fixing this week, and then figure out how > to do the other stuff after PGCon. Alternatively, we can let the > latest round of changes that are already in the tree settle unti

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread Geoff Winkless
On 8 June 2015 at 16:01, Robert Haas wrote: > On Mon, Jun 8, 2015 at 9:21 AM, Geoff Winkless > wrote: > > Wow! I never knew there were all these people out there who would be > rushing > > to help test if only the PG developers released alpha versions. It's > funny > > how they never used to do

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread Petr Jelinek
On Mon, Jun 8, 2015 at 5:01 , Robert Haas wrote: On Mon, Jun 8, 2015 at 9:21 AM, Geoff Winkless wrote: Wow! I never knew there were all these people out there who would be rushing to help test if only the PG developers released alpha versions. It's funny how they never used to do it when

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-08 Thread Amit Kapila
On Mon, Jun 8, 2015 at 6:39 PM, Andrew Dunstan wrote: > > On 06/08/2015 12:08 AM, Amit Kapila wrote: > >> How about if it is just a flat file with same name as tablespace link, >> why we want to give error for that case? I think now it just don't do >> anything with that file (unlink will fail w

Re: [HACKERS] bugfix: incomplete implementation of errhidecontext

2015-06-08 Thread Pavel Stehule
2015-06-08 16:49 GMT+02:00 Andres Freund : > On 2015-06-08 14:44:53 +, Jeevan Chalke wrote: > > The following review has been posted through the commitfest application: > > make installcheck-world: tested, passed > > Implements feature: tested, passed > > Spec compliant: teste

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread Joshua D. Drake
On 06/08/2015 06:21 AM, Geoff Winkless wrote: Wow! I never knew there were all these people out there who would be rushing to help test if only the PG developers released alpha versions. It's funny how they never used to do it when those alphas were done. The type of responses you are providi

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread Robert Haas
On Mon, Jun 8, 2015 at 9:21 AM, Geoff Winkless wrote: > Wow! I never knew there were all these people out there who would be rushing > to help test if only the PG developers released alpha versions. It's funny > how they never used to do it when those alphas were done. That's probably overplaying

Re: [HACKERS] bugfix: incomplete implementation of errhidecontext

2015-06-08 Thread Andres Freund
On 2015-06-08 14:44:53 +, Jeevan Chalke wrote: > The following review has been posted through the commitfest application: > make installcheck-world: tested, passed > Implements feature: tested, passed > Spec compliant: tested, passed > Documentation:tested, passed >

Re: [HACKERS] bugfix: incomplete implementation of errhidecontext

2015-06-08 Thread Jeevan Chalke
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed This is trivial bug fix in the area of hiding error context.

Re: [HACKERS] [COMMITTERS] pgsql: Add pg_audit, an auditing extension

2015-06-08 Thread Noah Misch
Ian, Abhijit, and David, My condemnation of the pg_audit commits probably hurt you as the feature's authors. I am sorry for that. Your code was better than most "Ready for Committer" code, and I hope you submit more patches in the future. Regards, nm -- Sent via pgsql-hackers mailing list (p

Re: [HACKERS] Collection of memory leaks for ECPG driver

2015-06-08 Thread Michael Paquier
On Mon, Jun 8, 2015 at 9:22 PM, Michael Meskes wrote: > On Mon, Jun 08, 2015 at 01:57:32PM +0900, Michael Paquier wrote: >> diff --git a/src/interfaces/ecpg/compatlib/informix.c >> b/src/interfaces/ecpg/compatlib/informix.c >> index d6de3ea..c1e3dfb 100644 >> --- a/src/interfaces/ecpg/compatlib/in

Re: [HACKERS] Memory leak fixes for pg_dump, pg_dumpall, initdb and pg_upgrade

2015-06-08 Thread Michael Paquier
On Mon, Jun 8, 2015 at 10:26 PM, Michael Paquier wrote: > On Mon, Jun 8, 2015 at 3:48 PM, Michael Paquier > wrote: >> Hi all, >> >> Please find attached a set of fixes for a couple of things in src/bin: >> - pg_dump/pg_dumpall: >> -- getFormattedTypeName, convertTSFunction and myFormatType return

Re: [HACKERS] Memory leak fixes for pg_dump, pg_dumpall, initdb and pg_upgrade

2015-06-08 Thread Michael Paquier
On Mon, Jun 8, 2015 at 3:48 PM, Michael Paquier wrote: > Hi all, > > Please find attached a set of fixes for a couple of things in src/bin: > - pg_dump/pg_dumpall: > -- getFormattedTypeName, convertTSFunction and myFormatType return > strdup'd results that are never free'd. > -- convertTSFunction

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread Geoff Winkless
Among several others, On 8 June 2015 at 13:59, David Gould wrote: > I think Alphas are valuable and useful and even more so if they have > release > notes. For example, some of my clients are capable of fetching sources and > building from scratch and filing bug reports and are often interested i

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-08 Thread Andres Freund
On 2015-06-05 16:56:18 -0400, Tom Lane wrote: > Andres Freund writes: > > On June 5, 2015 10:02:37 PM GMT+02:00, Robert Haas > > wrote: > >> I think we would be foolish to rush that part into the tree. We > >> probably got here in the first place by rushing the last round of > >> fixes too much

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-08 Thread Andrew Dunstan
On 06/08/2015 12:08 AM, Amit Kapila wrote: On Mon, Jun 8, 2015 at 5:52 AM, Andrew Dunstan > wrote: > > On 06/05/2015 11:08 PM, Amit Kapila wrote: >> >> >> Okay, I think I can understand why you want to be cautious for >> having a different check for this path,

[HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Robert Haas
Hi, I think we have consensus that we should proceed with releasing fixes for the known multixact bugs in two stages: - One set of minor releases with the fixes that we have now, to undo the damage caused by 9.4.2 and still present in 9.4.3. These changes will force immediate anti-wraparound vac

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread David Gould
I think Alphas are valuable and useful and even more so if they have release notes. For example, some of my clients are capable of fetching sources and building from scratch and filing bug reports and are often interested in particular new features. They even have staging infrastructure that could

Re: [HACKERS] Collection of memory leaks for ECPG driver

2015-06-08 Thread Michael Meskes
On Mon, Jun 08, 2015 at 01:57:32PM +0900, Michael Paquier wrote: > Please find attached a patch aimed at fixing a couple of memory leaks > in the ECPG driver. Coverity (and sometimes I after reading some other > code paths) found them. And some are not correct it seems. But then some of the code i

Re: [HACKERS] CREATE POLICY and RETURNING

2015-06-08 Thread Dean Rasheed
On 6 June 2015 at 22:16, Peter Geoghegan wrote: > On Fri, Oct 17, 2014 at 5:34 AM, Andres Freund wrote: >> On 2014-10-17 14:57:03 +0800, Craig Ringer wrote: >>> On 10/17/2014 02:49 AM, Robert Haas wrote: >>> > I think you could probably make the DELETE policy control what can get >>> > deleted, b

Re: [HACKERS] RLS fails to work with UPDATE ... WHERE CURRENT OF

2015-06-08 Thread Dean Rasheed
On 6 June 2015 at 23:28, Peter Geoghegan wrote: > Attached test case patch shows how RLS fails to play nice with UPDATE > ... WHERE CURRENT OF. > [snip] > What's actually occurring here is that the executor imagines that this > involves a foreign table scan (although I suppose it's equivocating a

Re: [HACKERS] pg_stat_archiver issue with aborted archiver

2015-06-08 Thread Julien Rouhaud
Le 08/06/2015 05:56, Michael Paquier a écrit : > On Sun, Jun 7, 2015 at 1:11 AM, Julien Rouhaud > wrote: >> I just noticed that if the archiver aborts (for instance if the >> archive_command exited with a return code > 127), >> pg_stat_archiver won't report those failed attempts. This happens >>

Re: [HACKERS] checkpointer continuous flushing

2015-06-08 Thread Fabien COELHO
Hello Cédric, It looks a bit hazardous, do you have a benchmark for freeBSD ? No, I just consulted the FreeBSD man page for posix_fadvise. I someone can run tests on something which HDDs is not linux, that would be nice. Sources says: case POSIX_FADV_DONTNEED: /*

Re: [HACKERS] checkpointer continuous flushing

2015-06-08 Thread Cédric Villemain
Le 07/06/2015 16:53, Fabien COELHO a écrit : > +» » /*·Others:·say·that·data·should·not·be·kept·in·memory... > +» » ·*·This·is·not·exactly·what·we·want·to·say,·because·we·want·to·write > +» » ·*·the·data·for·durability·but·we·may·need·it·later·nevertheless. > +» » ·*·It·seems·that·Linux·would·free·

  1   2   >