Re: [HACKERS] Let PostgreSQL's On Schedule checkpoint write buffer smooth spread cycle by tuning IsCheckpointOnSchedule?

On 2015-07-03 07:38:15 +0200, Fabien COELHO wrote: > I've submitted a patch to improve checkpoint write scheduling, including X00 > hours of performance test on various cases. This patch changes significantly > the load distribution over the whole checkpoint, and AFAICS has been tested > on rather

Re: [HACKERS] Memory leak fixes for pg_dump, pg_dumpall, initdb and pg_upgrade

On Fri, Jul 3, 2015 at 3:14 AM, Heikki Linnakangas wrote: > I committed some of these that seemed like improvements on readability > grounds, but please just mark the rest as "ignore" in coverity. Done. Thanks. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) T

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On 2015/07/02 23:13, Kouhei Kaigai wrote: > >> To be honest, ISTM that it's difficult to do that simply and efficiently > >> for the foreign/custom-join-pushdown API that we have for 9.5. It's a > >> little late, but what I started thinking is to redesign that API so that > >> that API is called

Re: [HACKERS] Let PostgreSQL's On Schedule checkpoint write buffer smooth spread cycle by tuning IsCheckpointOnSchedule?

On 3 July 2015 at 06:38, Fabien COELHO wrote: > > Hello Simon, > > We could do better, but that is not a reason not to commit this, as is. >> Commit, please. >> > > My 0,02€: Please do not commit without further testing... > > I've submitted a patch to improve checkpoint write scheduling, includ

Re: [HACKERS] PostgreSQL 9.5 Alpha 1 build fail with perl 5.22

On Fri, Jul 3, 2015 at 2:47 PM, Marco Atzeri wrote: > On 7/2/2015 5:16 PM, Dave Page wrote: >> >> The PostgreSQL Global Development Group announces that the alpha >> release of PostgreSQL 9.5, the latest version of the world's leading >> open source database, is available today. This release cont

Re: [HACKERS] WAL logging problem in 9.4.3?

On Fri, Jul 03, 2015 at 02:34:44PM +0900, Fujii Masao wrote: > > Hmm, for me it is 100% reproducable. Are you familiar with Docker? I > > can probably construct a Dockerfile that reproduces it pretty reliably. > > I could reproduce the problem in the master branch by doing > the following steps.

Re: [HACKERS] PostgreSQL 9.5 Alpha 1 build fail with perl 5.22

On Fri, Jul 3, 2015 at 2:47 PM, Marco Atzeri wrote: > On 7/2/2015 5:16 PM, Dave Page wrote: >> >> The PostgreSQL Global Development Group announces that the alpha >> release of PostgreSQL 9.5, the latest version of the world's leading >> open source database, is available today. This release cont

[HACKERS] PostgreSQL 9.5 Alpha 1 build fail with perl 5.22

On 7/2/2015 5:16 PM, Dave Page wrote: The PostgreSQL Global Development Group announces that the alpha release of PostgreSQL 9.5, the latest version of the world's leading open source database, is available today. This release contains previews of all of the features which will be available in t

Re: [HACKERS] Let PostgreSQL's On Schedule checkpoint write buffer smooth spread cycle by tuning IsCheckpointOnSchedule?

Hello Simon, We could do better, but that is not a reason not to commit this, as is. Commit, please. My 0,02€: Please do not commit without further testing... I've submitted a patch to improve checkpoint write scheduling, including X00 hours of performance test on various cases. This patch

Re: [HACKERS] WAL logging problem in 9.4.3?

On Fri, Jul 3, 2015 at 2:20 PM, Martijn van Oosterhout wrote: > On Fri, Jul 03, 2015 at 12:21:02AM +0200, Andres Freund wrote: >> Hi, >> >> On 2015-07-03 00:05:24 +0200, Martijn van Oosterhout wrote: >> > === Start with an empty database >> >> My guess is you have wal_level = minimal? > > Default

Re: [HACKERS] multivariate statistics / patch v7

Hello, I started to work on this patch. > attached is v7 of the multivariate stats patch. The main improvement > is major refactoring of the clausesel.c portion - splitting the > awfully long spaghetti-style functions into smaller pieces, making it > much more understandable etc. Thank you, it lo

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

On 2015-07-02 PM 11:00, Syed, Rahila wrote: > Hello, > >> Unless I am missing something, I guess you can still keep the actual code >> that updates counters outside the core if you adopt an approach that Simon >> suggests. > Yes. The code to extract progress information from VACUUM and storing i

Re: [HACKERS] WAL logging problem in 9.4.3?

On Fri, Jul 03, 2015 at 12:21:02AM +0200, Andres Freund wrote: > Hi, > > On 2015-07-03 00:05:24 +0200, Martijn van Oosterhout wrote: > > === Start with an empty database > > My guess is you have wal_level = minimal? Default config, was just initdb'd. So yes, the default wal_level = minimal. > >

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/07/02 23:13, Kouhei Kaigai wrote: To be honest, ISTM that it's difficult to do that simply and efficiently for the foreign/custom-join-pushdown API that we have for 9.5. It's a little late, but what I started thinking is to redesign that API so that that API is called at standard_join_se

Re: [HACKERS] Support for N synchronous standby servers - take 2

Josh Berkus wrote: > > Say you take this case: > > "2" : { "local_replica", "london_server", "nyc_server" } > > ... which should ensure that any data which is replicated is replicated > to at least two places, so that even if you lose the entire local > datacenter, you have the data on at least

Re: [HACKERS] PATCH: remove nclients/nthreads constraint from pgbench

This doesn't behave correctly if you set -j to greater than -c, and also use the rate limit option: Indeed. v3 attached fixed the case when nthreads > nclients. -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index a808546..2517a3a 100644 --- a/doc/src/sgm

Re: [HACKERS] bugfix: incomplete implementation of errhidecontext

2015-07-03 1:07 GMT+02:00 Tom Lane : > Andres Freund writes: > > On 2015-06-08 14:44:53 +, Jeevan Chalke wrote: > >> This is trivial bug fix in the area of hiding error context. > >> > >> I observed that there are two places from which we are calling this > function > >> to hide the context i

Re: [HACKERS] Synch failover WAS: Support for N synchronous standby servers - take 2

On Fri, Jul 3, 2015 at 6:54 AM, Josh Berkus wrote: > On 07/02/2015 12:44 PM, Andres Freund wrote: >> On 2015-07-02 11:50:44 -0700, Josh Berkus wrote: >>> So there's two parts to this: >>> >>> 1. I need to ensure that data is replicated to X places. >>> >>> 2. I need to *know* which places data was

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

On Thu, Jul 2, 2015 at 7:44 PM, Alvaro Herrera wrote: > > Amit Kapila wrote: > > > > Added the above log messages in attached patch with small change > > such that in message, file names will be displayed with quotes as most > > of other usages of rename (failure) in that file uses quotes to displ

Re: [HACKERS] WAL-related tools and .paritial WAL file

On Thu, Jul 2, 2015 at 8:59 PM, Michael Paquier wrote: > On Thu, Jul 2, 2015 at 8:42 PM, Fujii Masao wrote: >>> 3) Something not caused by this patch that I just noticed... But >>> pg_resetxlog does not remove .backup files in pg_xlog. Shouldn't they >>> get moved away as well? >> >> pg_resetxlog

[HACKERS] Interesting study "what is C in practice"

We spend a lot of time debating what non standard behaviours are safe to rely on because other software relies on it. Someone actually went and surveyed some major projects about what behaviours are being counted on. Three results aren't as conclusive as I would have expected but even so some of t

Re: [HACKERS] Exposing PG_VERSION_NUM in pg_config

On Fri, Jul 3, 2015 at 6:27 AM, Tom Lane wrote: > Michael Paquier writes: >> On Wed, Mar 25, 2015 at 8:26 AM, Tom Lane wrote: >> ... So attached is a patch that adds VERSION_NUM in >> Makefile.global. > > While there was not exactly universal consensus that we need this, the > patch as given is

Re: [HACKERS] bugfix: incomplete implementation of errhidecontext

Andres Freund writes: > On 2015-06-08 14:44:53 +, Jeevan Chalke wrote: >> This is trivial bug fix in the area of hiding error context. >> >> I observed that there are two places from which we are calling this function >> to hide the context in log messages. Those were broken. > Broken in whi

Re: [HACKERS] WAL logging problem in 9.4.3?

Hi, On 2015-07-03 00:05:24 +0200, Martijn van Oosterhout wrote: > === Start with an empty database My guess is you have wal_level = minimal? > ctmp=# begin; > BEGIN > ctmp=# create table test(id serial primary key); > CREATE TABLE > ctmp=# truncate table test; > TRUNCATE TABLE > ctmp=# commit; >

[HACKERS] Determine operator from it's function

Is there a way to determine the operator that resulted in calling the operator function? I thought fcinfo->flinfo->fn_expr might get set to the OpExpr, but seems it can be a FuncExpr even when called via an operator... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Data in Trou

Re: [HACKERS] pg_restore -t should match views, matviews, and foreign tables

Craig Ringer writes: > I think you're right that sequences should be included by pg_restore since > they are by pg_dump, though. So v3 patch attached. You forgot "SEQUENCE SET" :-(. I fixed that and adjusted the docs a bit more and committed. regards, tom lane -- Sent

Re: [HACKERS] Memory Accounting v11

On Thu, Jul 2, 2015 at 11:37 AM, CK Tan wrote: > > I am sorry to ask questions unrelated to the subject, but how is tracking > memory going to fix the HashAgg blow up problem? Is there a plan to make > HashAgg not blow up (i.e. spill the hash table)? > Your question is probably answered here: htt

[HACKERS] WAL logging problem in 9.4.3?

Hoi, I ran into this in our CI setup and I thinks it's an actual bug. The issue appears to be that when a table is created *and truncated* in a single transaction, that the WAL log is logging a truncate record it shouldn't, such that if the database crashes you have a broken index. It would also l

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

On Thu, Jul 2, 2015 at 10:49 AM, Peter Geoghegan wrote: > Well, waiting means getting a ShareLock on the other session's XID. > You can't do that without first releasing your locks, unless you're > okay with unprincipled deadlocks. Besides, if the other session wins the race and inserts a physica

Re: [HACKERS] Time to fully remove heap_formtuple() and friends?

On Thu, Jul 2, 2015 at 11:22 AM, Heikki Linnakangas wrote: > Ok, committed. Thanks. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Synch failover WAS: Support for N synchronous standby servers - take 2

On 07/02/2015 12:44 PM, Andres Freund wrote: > On 2015-07-02 11:50:44 -0700, Josh Berkus wrote: >> So there's two parts to this: >> >> 1. I need to ensure that data is replicated to X places. >> >> 2. I need to *know* which places data was synchronously replicated to >> when the master goes down. >

Re: [HACKERS] Exposing PG_VERSION_NUM in pg_config

Michael Paquier writes: > On Wed, Mar 25, 2015 at 8:26 AM, Tom Lane wrote: > ... So attached is a patch that adds VERSION_NUM in > Makefile.global. While there was not exactly universal consensus that we need this, the patch as given is merely two lines, so it seems awfully cheap to Just Do It.

Re: [HACKERS] Information of pg_stat_ssl visible to all users

On Thu, Jul 2, 2015 at 10:06 PM, Andres Freund wrote: > On 2015-07-02 16:52:01 -0300, Alvaro Herrera wrote: > > If there's interest in closing these holes, this might be a first > > I don't think such an isolated attempt buys us anything except maybe > unsatisfied users. > > I can see a benefit i

Re: [HACKERS] PATCH:do not set Win32 server-side socket buffer size on windows 2012

On 04/10/2015 01:46 PM, chenhj wrote: PostgreSQL set Win32 server-side socket buffer size to 32k since 2006, for performance reasons. While,on the newer version of Windows,such as windows 2012,the default socket buffer size is 64k, and seem has better performance(high throughput). So, i propos

Re: [HACKERS] Add checksums without --initdb

On 2015-07-02 23:43:17 +0300, Heikki Linnakangas wrote: > >You'd need, afaics, a bgworker that connects to every database to read > >pg_class, to figure out what type of page a relfilenode has. And this > >probably should call back into the relevant AM or such. > > Nah, we already assume that ever

Re: [HACKERS] Add checksums without --initdb

On 07/02/2015 11:28 PM, Andres Freund wrote: On 2015-07-02 22:53:40 +0300, Heikki Linnakangas wrote: Add a "enabling-checksums" mode to the server where it calculates checksums for anything it writes, but doesn't check or complain about incorrect checksums on reads. Put the server into that mode

Re: [HACKERS] [PATCH] two-arg current_setting() with fallback

Jeevan Chalke writes: > Attached patch which fixes my review comments. Applied with minor adjustments (mostly cosmetic, but did neither of you notice the compiler warning?) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Configurable location for extension .control files

02.07.2015, 20:31, Heikki Linnakangas kirjoitti: > On 03/04/2015 09:41 AM, Oskari Saarenmaa wrote: >> 18.02.2015, 01:49, Jim Nasby kirjoitti: >>> On 2/17/15 4:39 PM, Oskari Saarenmaa wrote: Here's a patch to allow overriding extension installation directory. The patch allows superusers to

Re: [HACKERS] Add checksums without --initdb

On 2015-07-02 22:53:40 +0300, Heikki Linnakangas wrote: > On 07/02/2015 10:39 PM, David Christensen wrote: > >Possible concerns here are whether checksums are included in WAL > >full_page_writes or if they are independently calculated; if the > >latter I think we’d be fine. If checksums are all ha

Re: [HACKERS] Information of pg_stat_ssl visible to all users

On 2015-07-02 16:52:01 -0300, Alvaro Herrera wrote: > If there's interest in closing these holes, this might be a first I don't think such an isolated attempt buys us anything except maybe unsatisfied users. I can see a benefit in allowing to restrict information about users and such in other clu

Re: [HACKERS] Add checksums without --initdb

On 07/02/2015 10:39 PM, David Christensen wrote: Possible concerns here are whether checksums are included in WAL full_page_writes or if they are independently calculated; if the latter I think we’d be fine. If checksums are all handled at the layer below WAL than any streamed/processed changes

Re: [HACKERS] Information of pg_stat_ssl visible to all users

Magnus Hagander wrote: > On Thu, Jul 2, 2015 at 5:40 PM, Peter Eisentraut wrote: > > Actually, I think the whole view shouldn't be accessible to unprivileged > > users, except maybe your own row. > > > I could go for some of the others if we think there's reason, but I don't > understand the dn p

Re: [HACKERS] Add checksums without --initdb

On 07/02/2015 12:39 PM, David Christensen wrote: > So on #postgresql, I was musing about methods of getting checksums > enabled/disabled without requiring a separate initdb step and minimizing the > downtime required to get such functionality enabled. Funny, I was thinking just yesterday about h

Re: [HACKERS] Support for N synchronous standby servers - take 2

On 2015-07-02 11:50:44 -0700, Josh Berkus wrote: > So there's two parts to this: > > 1. I need to ensure that data is replicated to X places. > > 2. I need to *know* which places data was synchronously replicated to > when the master goes down. > > My entire point is that (1) alone is useless un

Re: [HACKERS] Faster setup_param_list() in plpgsql

I wrote: > What this patch does is to remove setup_param_list() overhead for the > common case of PLPGSQL_DTYPE_VAR variables (ie, any non-composite type). > It does that by the expedient of keeping the ParamExternData image of such > a variable valid at all times. That adds a few cycles to assign

[HACKERS] Add checksums without --initdb

So on #postgresql, I was musing about methods of getting checksums enabled/disabled without requiring a separate initdb step and minimizing the downtime required to get such functionality enabled. What about adapting pg_basebackup to add the following options: -k|--checksums - build the replica

Re: [HACKERS] Improve testing notes?

On Thu, Jul 2, 2015 at 11:52 AM, Josh Berkus wrote: > I've added the following section to "how to beta test": > > https://wiki.postgresql.org/wiki/HowToBetaTest#Particular_Things_to_Test_in_9.5 > > It would be great if other folks could add particular things they'd like > to see users test, so tha

Re: [HACKERS] Information of pg_stat_ssl visible to all users

On Thu, Jul 2, 2015 at 5:40 PM, Peter Eisentraut wrote: > On 6/10/15 2:17 AM, Magnus Hagander wrote: > > AIUI that one was just about the DN field, and not about the rest. If I > > understand you correctly, you are referring to the whole thing, not just > > one field? > > I think at least the DN

[HACKERS] Improve testing notes?

Hackers: I've added the following section to "how to beta test": https://wiki.postgresql.org/wiki/HowToBetaTest#Particular_Things_to_Test_in_9.5 It would be great if other folks could add particular things they'd like to see users test, so that we can make the alpha/beta cycle as fast and fruitf

Re: [HACKERS] Support for N synchronous standby servers - take 2

On 07/02/2015 11:31 AM, Andres Freund wrote: > On 2015-07-02 11:10:27 -0700, Josh Berkus wrote: >> If we're always going to be polling the replicas for furthest ahead, >> then why bother implementing quorum synch at all? That's the basic >> question I'm asking. What does it buy us that we don't al

Re: [HACKERS] Memory Accounting v11

On 14 June 2015 at 23:51, Tomas Vondra wrote: > The current state, where HashAgg just blows up the memory, is just not >>> reasonable, and we need to track the memory to fix that problem. >>> >> >> Meh. HashAgg could track its memory usage without loading the entire >> system with a penalty. >>

Re: [HACKERS] Support for N synchronous standby servers - take 2

On 2015-07-02 11:10:27 -0700, Josh Berkus wrote: > If we're always going to be polling the replicas for furthest ahead, > then why bother implementing quorum synch at all? That's the basic > question I'm asking. What does it buy us that we don't already have? What do those topic have to do with e

Re: [HACKERS] Rework the way multixact truncations work

On 2015-07-02 13:58:45 -0400, Robert Haas wrote: > On Thu, Jul 2, 2015 at 11:52 AM, Robert Haas wrote: > > Will look at 0003 next. > > +appendStringInfo(buf, "offsets [%u, %u), members [%u, %u)", > > I don't think we typically use this style for notating intervals. I don't think we real

[HACKERS] [PATCH v1] GSSAPI encryption support

Hello -hackers, As previously discussed on this list, I have coded up GSSAPI encryption support. If it is easier for anyone, this code is also available for viewing on my github: https://github.com/postgres/postgres/compare/master...frozencemetery:feature/gssencrypt Fallback support is present i

Re: [HACKERS] Time to fully remove heap_formtuple() and friends?

On 06/13/2015 07:10 AM, Peter Geoghegan wrote: On Fri, Jun 12, 2015 at 8:47 PM, Tom Lane wrote: Peter Geoghegan writes: Attached patch actually removes heap_formtuple() and friends. Does this seem worthwhile? Seems reasonable, but at this point I would say this is 9.6 material; third-party-

Re: [HACKERS] Reducing ClogControlLock contention

On Wed, Jul 1, 2015 at 6:21 AM, Andres Freund wrote: > On 2015-07-01 11:19:40 +0100, Simon Riggs wrote: >> What "tricks" are being used?? >> >> Please explain why taking 2 locks is bad here, yet works fine elsewhere. > > I didn't say anything about 'bad'. It's more complicated than one > lock. Sud

Re: [HACKERS] Memory leak fixes for pg_dump, pg_dumpall, initdb and pg_upgrade

On 06/08/2015 09:48 AM, Michael Paquier wrote: Hi all, Please find attached a set of fixes for a couple of things in src/bin: - pg_dump/pg_dumpall: -- getFormattedTypeName, convertTSFunction and myFormatType return strdup'd results that are never free'd. -- convertTSFunction returns const char.

Re: [HACKERS] Support for N synchronous standby servers - take 2

On 07/01/2015 11:12 PM, Fujii Masao wrote: > I don't think this is good approach because there can be the case where > you need to promote even the standby server not having sync flag. > Please imagine the case where you have sync and async standby servers. > When the master goes down, the async st

Re: [HACKERS] Rework the way multixact truncations work

On Thu, Jul 2, 2015 at 11:52 AM, Robert Haas wrote: > Will look at 0003 next. +appendStringInfo(buf, "offsets [%u, %u), members [%u, %u)", I don't think we typically use this style for notating intervals. case XLOG_MULTIXACT_CREATE_ID: id = "CREATE_ID";

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

On Thu, Jul 2, 2015 at 1:30 AM, Heikki Linnakangas wrote: >> Sure, if a speculative inserter detects a conflict, it still has to >> wait. But not in the aminsert call, and not until it cleans up its >> physical insertion (by super-deleting). Clearly a >> CHECK_UNIQUE_SPECULATIVE would have to be m

Re: [HACKERS] Configurable location for extension .control files

On 03/04/2015 09:41 AM, Oskari Saarenmaa wrote: 18.02.2015, 01:49, Jim Nasby kirjoitti: On 2/17/15 4:39 PM, Oskari Saarenmaa wrote: 10.06.2013, 17:51, Dimitri Fontaine kirjoitti: Andres Freund writes: In any case, no packager is going to ship an insecure-by-default configuration, which is wh

Re: [HACKERS] Doubt about AccessExclusiveLock in ALTER TABLE .. SET ( .. );

The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested Looks functionally complete Need a test to show that ALTER TABLE works on vi

Re: [HACKERS] Freeze avoidance of very large table.

On 2 July 2015 at 16:30, Sawada Masahiko wrote: > Also, the flags of each heap page header might be set PD_ALL_FROZEN, > as well as all-visible > Is it possible to have VM bits set to frozen but not visible? The description makes those two states sound independent of each other. Are they? Or

Re: [HACKERS] Fix autoconf deprecation warnings

On 05/31/2015 05:22 AM, Andreas Karlsson wrote: I have attached new versions which apply on the current master. Thanks, applied. --- a/configure.in +++ b/configure.in @@ -1473,7 +1473,7 @@ if test x"$pgac_cv_func_sigsetjmp" = x"yes"; then AC_DEFINE(HAVE_SIGSETJMP, 1, [Define to 1 if you ha

Re: [HACKERS] [PATCH] Reload SSL certificates on SIGHUP

On 5/30/15 10:14 PM, Andreas Karlsson wrote: > I have written a patch which makes it possible to change SSL > certificates (and other SSL parameters, including the CRL) without > restarting PostgreSQL. In fact this patch also makes it possible to turn > on or off ssl entirely without restart. It do

Re: [HACKERS] Raising our compiler requirements for 9.6

Robert Haas writes: > So far this thread is all about the costs of desupporting compilers > that don't have these features, and you're making a good argument > (that I think we all agree with) that the cost is small. But you > haven't really said what the benefits are. I made the same remark wit

Re: [HACKERS] Raising our compiler requirements for 9.6

On 2015-07-02 11:46:25 -0400, Robert Haas wrote: > In the case of static inline, the main problem with the status quo > AFAICS is that you have to do a little dance any time you'd otherwise > use a static inline function (for those following along at home, "git > grep INCLUDE_DEFINITIONS src/includ

Re: [HACKERS] Rework the way multixact truncations work

On Mon, Jun 29, 2015 at 3:48 PM, Andres Freund wrote: > New version attached. 0002 looks good, but the commit message should perhaps mention the comment fix. Or commit that separately. Will look at 0003 next. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL C

Re: [HACKERS] Raising our compiler requirements for 9.6

On Wed, Jul 1, 2015 at 6:24 PM, Andres Freund wrote: > On 2015-07-02 00:15:14 +0200, Andres Freund wrote: >> animal OS compiler inline quiet inline >>varargs > >> brolga cygwin gcc-4.3 yy > > 4.3 obviously supports

Re: [HACKERS] raw output from copy

On 07/02/2015 11:02 AM, Tom Lane wrote: Andrew Dunstan writes: Does the COPY line protocol even support binary data? The protocol, per se, just transmits a byte stream. There is a field in the CopyInResponse/CopyOutResponse messages that indicates whether a text or binary copy is being done.

Re: [HACKERS] Information of pg_stat_ssl visible to all users

On 6/10/15 2:17 AM, Magnus Hagander wrote: > AIUI that one was just about the DN field, and not about the rest. If I > understand you correctly, you are referring to the whole thing, not just > one field? I think at least the DN field shouldn't be visible to unprivileged users. Actually, I think

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Jul 2, 2015 at 1:06 AM, Fujii Masao wrote: > On Thu, Jul 2, 2015 at 12:13 AM, Sawada Masahiko > wrote: >> On Thu, May 28, 2015 at 11:34 AM, Sawada Masahiko >> wrote: >>> On Thu, Apr 30, 2015 at 8:07 PM, Sawada Masahiko >>> wrote: On Fri, Apr 24, 2015 at 11:21 AM, Sawada Masahiko

Re: [HACKERS] raw output from copy

Andrew Dunstan writes: > Does the COPY line protocol even support binary data? The protocol, per se, just transmits a byte stream. There is a field in the CopyInResponse/CopyOutResponse messages that indicates whether a text or binary copy is being done. One thing we'd have to consider is wheth

Re: [HACKERS] raw output from copy

On 07/02/2015 10:07 AM, Pavel Stehule wrote: 2015-07-02 16:02 GMT+02:00 Andrew Dunstan >: On 07/02/2015 09:43 AM, Simon Riggs wrote: On 2 July 2015 at 14:02, Andrew Dunstan mailto:and...@dunslane.net>

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

"Syed, Rahila" writes: > Hello, >> Unless I am missing something, I guess you can still keep the actual code >> that updates counters outside the core if you adopt an approach that Simon >> suggests. > Yes. The code to extract progress information from VACUUM and storing in > shared memory can

Re: [HACKERS] raw output from copy

On 2 July 2015 at 15:07, Pavel Stehule wrote: > It can be used from psql without any problems. > It can, but your patch does not yet do that, while Andrew's does. We want a solution that works from psql and other clients. Hopefully the same-ish solution. -- Simon Riggshttp://

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> > Let me introduce a few cases we should pay attention. > > > > Foreign/CustomScan node may stack; that means a Foreign/CustomScan node > > may have child node that includes another Foreign/CustomScan node with > > scanrelid==0. > > (At this moment, ForeignScan cannot have child node, however, mo

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

Amit Kapila wrote: > On Sat, Jun 27, 2015 at 12:54 AM, Robert Haas wrote: > > > > On Mon, Jun 15, 2015 at 2:52 PM, Amit Kapila > > wrote: > > > Attached patch provides a fix as per above discussion. > > > > I think we should emit some LOG messages here. When we detect the > > file is there: > >

Re: [HACKERS] [BUGS] BUG #13126: table constraint loses its comment

On 2015-05-27 15:10, Michael Paquier wrote: On Wed, Apr 29, 2015 at 1:30 PM, Michael Paquier wrote: On Sun, Apr 26, 2015 at 6:05 AM, Tom Lane wrote: x...@resolvent.net writes: In some circumstances, the comment on a table constraint disappears. Here is an example: Hm, yeah. The problem i

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

Simon Riggs writes: > On 2 July 2015 at 14:08, Heikki Linnakangas wrote: >> I'm marking this as "returned with feedback" in the commitfest. > There are no unresolved issues with the approach, nor is it true it is > slower. If you think there are some, you should say what they are, not act > high

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

On Thu, Jul 2, 2015 at 7:27 PM, Heikki Linnakangas wrote: > > On 07/02/2015 04:18 PM, Amit Kapila wrote: >> >> >> Don't you think the approach discussed (delayed cleanup of buffers >> during checkpoint scan) is sufficient to fix the issues raised. > > > Dunno, but there is no patch for that. > Th

Re: [HACKERS] raw output from copy

2015-07-02 16:02 GMT+02:00 Andrew Dunstan : > > On 07/02/2015 09:43 AM, Simon Riggs wrote: > >> On 2 July 2015 at 14:02, Andrew Dunstan > and...@dunslane.net>> wrote: >> >> >> Please don't top-post on the PostgreSQL lists. You've been around >> here long enough to know that bottom posting

Re: [HACKERS] Reducing the size of BufferTag & remodeling forks

On 2015-07-02 09:51:59 -0400, Tom Lane wrote: > Andres Freund writes: > > 1) Introduce a shared pg_relfilenode table. Every table, even > >shared/nailed ones, get an entry therein. It's there to make it > >possibly to uniquely allocate relfilenodes across databases & > >tablespaces. >

Re: [HACKERS] raw output from copy

2015-07-02 15:43 GMT+02:00 Simon Riggs : > On 2 July 2015 at 14:02, Andrew Dunstan wrote: > >> >> Please don't top-post on the PostgreSQL lists. You've been around here >> long enough to know that bottom posting is our custom. >> >> I posted a patch for this in 2013 at < >> http://www.postgresql.

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

Hello, >Unless I am missing something, I guess you can still keep the actual code that >updates counters outside the core if you adopt an approach that Simon suggests. Yes. The code to extract progress information from VACUUM and storing in shared memory can be outside core even with pg_stat_act

Re: [HACKERS] raw output from copy

On 07/02/2015 09:43 AM, Simon Riggs wrote: On 2 July 2015 at 14:02, Andrew Dunstan > wrote: Please don't top-post on the PostgreSQL lists. You've been around here long enough to know that bottom posting is our custom. I posted a patch for this in 2013 a

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

On 07/02/2015 04:33 PM, Simon Riggs wrote: On 2 July 2015 at 14:08, Heikki Linnakangas wrote: On 06/27/2015 07:45 AM, Amit Kapila wrote: Sometime back on one of the PostgreSQL blog [1], there was discussion about the performance of drop/truncate table for large values of shared_buffers and i

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

On 07/02/2015 04:18 PM, Amit Kapila wrote: On Thu, Jul 2, 2015 at 6:38 PM, Heikki Linnakangas wrote: On 06/27/2015 07:45 AM, Amit Kapila wrote: Sometime back on one of the PostgreSQL blog [1], there was discussion about the performance of drop/truncate table for large values of shared_buffer

Re: [HACKERS] Reducing the size of BufferTag & remodeling forks

Andres Freund writes: > 1) Introduce a shared pg_relfilenode table. Every table, even >shared/nailed ones, get an entry therein. It's there to make it >possibly to uniquely allocate relfilenodes across databases & >tablespaces. > 2) Replace relation forks, with the exception of the ini

Re: [HACKERS] Support for N synchronous standby servers - take 2

On 2 July 2015 at 09:44, Beena Emerson wrote: > I am not sure how combining both will work out. > Use cases needed. > Catalog Method: > Is it safe to assume we may not going ahead with the Catalog approach? > Yes -- Simon Riggshttp://www.2ndQuadrant.com/

Re: [HACKERS] WALWriter active during recovery

On 2 July 2015 at 14:38, Andres Freund wrote: > On 2015-07-02 14:34:48 +0100, Simon Riggs wrote: > > This was pushed back from last CF and I haven't worked on it at all, nor > > will I. > > > > Pushing back again. > > Let's "return with feedback", not " move", it then.. Moving a entries > along w

Re: [HACKERS] Patch to improve a few appendStringInfo* calls

David Rowley wrote: > On 2 July 2015 at 21:59, Heikki Linnakangas wrote: > > > I left out the changes like > > > > - appendStringInfoString(&collist, buf.data); > >> + appendBinaryStringInfo(&collist, buf.data, buf.len); > >> > > > > because they're not an improvement

Re: [HACKERS] raw output from copy

On 2 July 2015 at 14:02, Andrew Dunstan wrote: > > Please don't top-post on the PostgreSQL lists. You've been around here > long enough to know that bottom posting is our custom. > > I posted a patch for this in 2013 at < > http://www.postgresql.org/message-id/50f2fa92.9040...@dunslane.net> but >

Re: [HACKERS] WALWriter active during recovery

On 2015-07-02 14:34:48 +0100, Simon Riggs wrote: > This was pushed back from last CF and I haven't worked on it at all, nor > will I. > > Pushing back again. Let's "return with feedback", not " move", it then.. Moving a entries along which aren't expected to receive updates anytime soon isn't a g

[HACKERS] Reducing the size of BufferTag & remodeling forks

Hi, I've complained a number of times that our BufferTag is ridiculously large: typedef struct buftag { RelFileNode rnode; /* physical relation identifier */ ForkNumber forkNum; BlockNumber blockNum; /* blknum relative to begin of reln */ } BufferTag; typedef struct Re

Re: [HACKERS] WALWriter active during recovery

On 2 July 2015 at 14:31, Fujii Masao wrote: > On Thu, Mar 5, 2015 at 5:22 PM, Fujii Masao wrote: > > On Thu, Dec 18, 2014 at 6:43 PM, Fujii Masao > wrote: > >> On Tue, Dec 16, 2014 at 3:51 AM, Simon Riggs > wrote: > >>> Currently, WALReceiver writes and fsyncs data it receives. Clearly, > >>>

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

On 2 July 2015 at 14:08, Heikki Linnakangas wrote: > On 06/27/2015 07:45 AM, Amit Kapila wrote: > >> Sometime back on one of the PostgreSQL blog [1], there was >> discussion about the performance of drop/truncate table for >> large values of shared_buffers and it seems that as the value >> of sha

Re: [HACKERS] WALWriter active during recovery

On Thu, Mar 5, 2015 at 5:22 PM, Fujii Masao wrote: > On Thu, Dec 18, 2014 at 6:43 PM, Fujii Masao wrote: >> On Tue, Dec 16, 2014 at 3:51 AM, Simon Riggs wrote: >>> Currently, WALReceiver writes and fsyncs data it receives. Clearly, >>> while we are waiting for an fsync we aren't doing any other

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

On Thu, Jul 2, 2015 at 6:38 PM, Heikki Linnakangas wrote: > > On 06/27/2015 07:45 AM, Amit Kapila wrote: >> >> Sometime back on one of the PostgreSQL blog [1], there was >> discussion about the performance of drop/truncate table for >> large values of shared_buffers and it seems that as the value

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

On 2015-07-02 16:08:03 +0300, Heikki Linnakangas wrote: > I'm marking this as "returned with feedback" in the commitfest. In addition > to the issues raised so far, ISTM that the patch makes dropping a very large > table with small shared_buffers slower (DropForkSpecificBuffers() is O(n) > where n

  1   2   >