Re: [HACKERS] Declarative partitioning

On Sat, Feb 20, 2016 at 1:41 PM, Peter Eisentraut wrote: > On 2/16/16 9:56 PM, Amit Langote wrote: >> From now on, instead of attaching multiple files like in the previous >> message, I will send a single tar.gz which will contain patches created by >> git-format-patch. > > Please don't do that.

Re: [HACKERS] [JDBC] JDBC behaviour

Hi All I understand your point, may be I didn't understand everyone or everyone didn't understand me one feature of PostgreSQL is implemented into another feature of Java ( i say subject PostgreSQL::autocommit Vs JDBC::setAutoCommit ), i.e PostgreSQL::"set autocommit to FALSE" is implemented as

Re: [HACKERS] Declarative partitioning

On 2/16/16 9:56 PM, Amit Langote wrote: > From now on, instead of attaching multiple files like in the previous > message, I will send a single tar.gz which will contain patches created by > git-format-patch. Please don't do that. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

[HACKERS] GCC 6 warning fixes

Here are three patches to fix new warnings in GCC 6. 0001 is apparently a typo. 0002 was just (my?) stupid code to begin with. 0003 is more of a workaround. There could be other ways address this, too. From 1e5bf0bdcd86b807d881ea82245275389083ec75 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut

Re: [HACKERS] exposing pg_controldata and pg_config as functions

On 01/17/2016 04:10 PM, Joe Conway wrote: > On 01/16/2016 06:02 AM, Michael Paquier wrote: >> On Wed, Dec 30, 2015 at 9:08 AM, Joe Conway wrote: >>> 3) Adds new functions, more or less in line with previous discussions: >>>* pg_checkpoint_state() >>>* pg_controldata_state() >>>* pg_rec

[HACKERS] psql metaqueries with \gexec

Often, I'm faced with a long .sql script that builds some objects, then builds things on top of them. This means that some of the queries I wish to run are dependent on the state of things that are unknown at the time of writing the script. I could give up, and make a python script that mostly ju

Re: [HACKERS] checkpointer continuous flushing - V16

On Sat, Feb 20, 2016 at 5:08 AM, Fabien COELHO wrote: >> Kernel 3.2 is extremely bad for Postgresql, as the vm seems to amplify IO >> somehow. The difference to 3.13 (the latest LTS kernel for 12.04) is huge. >> >> >> https://medium.com/postgresql-talk/benchmarking-postgresql-with-different-linux-

Re: [HACKERS] checkpointer continuous flushing - V18

On 2016-02-19 22:46:44 +0100, Fabien COELHO wrote: > > Hello Andres, > > >Here's the next two (the most important) patches of the series: > >0001: Allow to trigger kernel writeback after a configurable number of > >writes. > >0002: Checkpoint sorting and balancing. > > I will look into these tw

Re: [HACKERS] checkpointer continuous flushing - V18

Hello Andres, Here's the next two (the most important) patches of the series: 0001: Allow to trigger kernel writeback after a configurable number of writes. 0002: Checkpoint sorting and balancing. I will look into these two in depth. Note that I would have ordered them in reverse because sor

Re: [HACKERS] proposal: make NOTIFY list de-duplication optional

On 2/9/16, Tom Lane wrote: > FWIW, I think it would be a good thing if the NOTIFY statement syntax were > not remarkably different from the syntax used in the pg_notify() function > call. To do otherwise would certainly be confusing. So on the whole > I'd go with the "NOTIFY channel [ , payload

Re: [HACKERS] proposal: PL/Pythonu - function ereport

2016-02-18 17:59 GMT+01:00 Catalin Iacob : > On 2/18/16, Pavel Stehule wrote: > > it doesn't look badly. Is there any possibility how to emulate it with > > Python2 ? What do you think about some similar implementation on Python2? > > The example code I gave works as is in Python2. > > The Python

Re: [HACKERS] pg_ctl promote wait

Andres Freund writes: > On 2016-02-19 15:09:58 -0500, Tom Lane wrote: >> I see no need for an additional mechanism. Just watch pg_control until >> you see DB_IN_PRODUCTION state there, then switch over to the same >> connection probing that "pg_ctl start -w" uses. > That's afaics not sufficient

Re: [HACKERS] pg_ctl promote wait

On 2016-02-19 15:09:58 -0500, Tom Lane wrote: > Peter Eisentraut writes: > > On 2/19/16 10:06 AM, Fujii Masao wrote: > >> One concern is that there can be a "time" after the pg_control's state > >> is changed to DB_IN_PRODUCTION and before the server is able to > >> start accepting normal (not rea

Re: [HACKERS] checkpointer continuous flushing - V18

On 2016-02-04 16:54:58 +0100, Andres Freund wrote: > Hi, > > Fabien asked me to post a new version of the checkpoint flushing patch > series. While this isn't entirely ready for commit, I think we're > getting closer. > > I don't want to post a full series right now, but my working state is > ava

Re: [HACKERS] FDW: should GetFdwRoutine be called when drop table?

Andres Freund writes: > On 2016-02-19 14:18:19 -0500, Peter Eisentraut wrote: >> On 2/19/16 12:21 PM, Feng Tian wrote: >>> I have an fdw that each foreign table will acquire some persisted resource. >> But foreign data wrappers are meant to be wrappers around data managed >> elsewhere, not their

Re: [HACKERS] Declarative partitioning

On Thu, Feb 18, 2016 at 12:41 AM, Amit Langote < langote_amit...@lab.ntt.co.jp> wrote: > START [ EXCL ] (startval) END [ INCL ] (endval) > > That is, in range type notation, '[startval, endval)' is the default > behavior. So for each partition, there is at least the following pieces of > metadata:

Re: [HACKERS] pg_ctl promote wait

Peter Eisentraut writes: > On 2/19/16 10:06 AM, Fujii Masao wrote: >> One concern is that there can be a "time" after the pg_control's state >> is changed to DB_IN_PRODUCTION and before the server is able to >> start accepting normal (not read-only) connections. So if users try to >> start write t

Re: [HACKERS] checkpointer continuous flushing - V16

Hallo Patric, Kernel 3.2 is extremely bad for Postgresql, as the vm seems to amplify IO somehow. The difference to 3.13 (the latest LTS kernel for 12.04) is huge. https://medium.com/postgresql-talk/benchmarking-postgresql-with-different-linux-kernel-versions-on-ubuntu-lts-e61d57b70dd4#.6dx44

Re: [HACKERS] pg_ctl promote wait

Peter Eisentraut writes: > Is it safe to read pg_control externally without a lock? pg_controldata > will just report a CRC error and proceed, and if you're not sure you can > just run it again. But if a promotion fails every so often because of > concurrent pg_control writes, that would make th

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

On Fri, Feb 19, 2016 at 12:05 PM, Alvaro Herrera wrote: > But those pages are supposed to be used as the index grows. So unless > they are forgotten by the FSM, they shouldn't accumulate. (Except where > the table doesn't grow but only shrinks, so there's no need for new > index pages, but I don

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

Peter Geoghegan wrote: > On Thu, Feb 18, 2016 at 4:53 PM, Tom Lane wrote: > >> there are usage patterns where half-dead pages might accumulate. > > > > Other than a usage pattern of "randomly SIGKILL backends every few > > seconds", I don't see how that would happen. > > I meant where pages coul

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

On Thu, Feb 18, 2016 at 4:53 PM, Tom Lane wrote: > Only a physical-order scan, ie vacuum, would visit a dead page > (ignoring transient corner cases like a page getting deleted while an > indexscan is in flight to it). So I think treating it as part of the > fragmentation measure is completely wr

Re: [HACKERS] Restructuring Paths to allow per-Path targetlist info

Alvaro Herrera writes: >> So, the attached patch just bites the bullet and adds explicit output >> tlist information to struct Path. > Hmm, I wonder if this can be used to attack the problem here in a more > sensible manner: > https://github.com/2ndQuadrant/postgres/commit/e7c5df6b614b542d55588a4

Re: [HACKERS] FDW: should GetFdwRoutine be called when drop table?

On 2016-02-19 14:18:19 -0500, Peter Eisentraut wrote: > On 2/19/16 12:21 PM, Feng Tian wrote: > > I have an fdw that each foreign table will acquire some persisted resource. > > In my case, some files in file system. To drop the table cleanly, I > > have written > > an object_access_hook that rem

Re: [HACKERS] pg_ctl promote wait

On 2016-02-20 00:06:09 +0900, Fujii Masao wrote: > One concern is that there can be a "time" after the pg_control's state > is changed to DB_IN_PRODUCTION and before the server is able to > start accepting normal (not read-only) connections. So if users try to > start write transaction just after p

Re: [HACKERS] FDW: should GetFdwRoutine be called when drop table?

On 2/19/16 12:21 PM, Feng Tian wrote: > I have an fdw that each foreign table will acquire some persisted resource. > In my case, some files in file system. To drop the table cleanly, I > have written > an object_access_hook that remove those files. The hook is installed in > _PG_init. > > It

Re: [HACKERS] pg_ctl promote wait

On 2016-02-19 13:48:52 -0500, Peter Eisentraut wrote: > Is it safe to read pg_control externally without a lock? pg_controldata > will just report a CRC error and proceed, and if you're not sure you can > just run it again. But if a promotion fails every so often because of > concurrent pg_contro

Re: [HACKERS] pg_ctl promote wait

On 2/19/16 10:06 AM, Fujii Masao wrote: > One concern is that there can be a "time" after the pg_control's state > is changed to DB_IN_PRODUCTION and before the server is able to > start accepting normal (not read-only) connections. So if users try to > start write transaction just after pg_ctl pro

Re: [HACKERS] pg_ctl promote wait

On 2/18/16 3:33 AM, Andres Freund wrote: > Hi, > > On 2016-02-17 21:47:50 -0500, Peter Eisentraut wrote: >> It would be nice if pg_ctl promote supported the -w (wait) option. >> >> How could pg_ctl determine when the promotion has finished? > > How about looking into pg_control? ControlFileData->

Re: [HACKERS] about google summer of code 2016

On 02/19/2016 10:10 AM, Álvaro Hernández Tortosa wrote: Hi. Oleg and I discussed recently that a really good addition to a GSoC item would be to study whether it's convenient to have a binary serialization format for jsonb over the wire. Some argue this should be benchmarked first. So

Re: [HACKERS] checkpointer continuous flushing - V16

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Fabien, Fabien COELHO schrieb am 19.02.2016 um 16:04: > >>> [...] Ubuntu 12.04 LTS (precise) >> >> That's with 12.04's standard kernel? > > Yes. Kernel 3.2 is extremely bad for Postgresql, as the vm seems to amplify IO somehow. The difference

Re: [HACKERS] about google summer of code 2016

Hi. Oleg and I discussed recently that a really good addition to a GSoC item would be to study whether it's convenient to have a binary serialization format for jsonb over the wire. Some argue this should be benchmarked first. So the scope for this project would be to benchmark and a

[HACKERS] Google Summer of Code

Mentoring organization application deadline is 19 February 19:00 UTC

[HACKERS] FDW: should GetFdwRoutine be called when drop table?

Hi, Hackers, I have an fdw that each foreign table will acquire some persisted resource. In my case, some files in file system. To drop the table cleanly, I have written an object_access_hook that remove those files. The hook is installed in _PG_init. It all worked well except one case. Suppos

Re: [HACKERS] PostgreSQL Audit Extension

On Fri, Feb 19, 2016 at 08:20:31AM -0800, Andres Freund wrote: > On 2016-02-19 10:14:47 -0500, Bruce Momjian wrote: > > We have already hesitated to record DDL changes for > > logical replication because of the code size, maintenance overhead, and > > testing required. > > I'm not sure what you're

Re: [HACKERS] PostgreSQL Audit Extension

On Fri, Feb 19, 2016 at 11:20:13AM -0500, David Steele wrote: > On 2/19/16 10:54 AM, Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > >> Understood. My point is that there is a short list of read events, and > >> many DDL events. We have already hesitated to record DDL changes for > >> logic

Re: [HACKERS] about google summer of code 2016

Atri Sharma wrote: > I agree, there might be scope for non core projects and PL/Java sounds like > a good area. We've hosted MADlib-based projects in the past, so why not. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & S

Re: [HACKERS] PostgreSQL Audit Extension

On 2016-02-19 10:14:47 -0500, Bruce Momjian wrote: > We have already hesitated to record DDL changes for > logical replication because of the code size, maintenance overhead, and > testing required. I'm not sure what you're referring to here? It'd be some relatively minor code surgery to also pass

Re: [HACKERS] PostgreSQL Audit Extension

On 2/19/16 10:54 AM, Alvaro Herrera wrote: > Bruce Momjian wrote: > >> Understood. My point is that there is a short list of read events, and >> many DDL events. We have already hesitated to record DDL changes for >> logical replication because of the code size, maintenance overhead, and >> test

Re: [HACKERS] PostgreSQL Audit Extension

On Fri, Feb 19, 2016 at 12:54:17PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > Understood. My point is that there is a short list of read events, and > > many DDL events. We have already hesitated to record DDL changes for > > logical replication because of the code size, maintenan

Re: [HACKERS] PostgreSQL Audit Extension

Bruce Momjian wrote: > Understood. My point is that there is a short list of read events, and > many DDL events. We have already hesitated to record DDL changes for > logical replication because of the code size, maintenance overhead, and > testing required. DDL is already captured using the ev

Re: [HACKERS] Restructuring Paths to allow per-Path targetlist info

Tom Lane wrote: > So, the attached patch just bites the bullet and adds explicit output > tlist information to struct Path. I did set things up so that the cost > is only one pointer in each Path in the typical case where Paths emit > the set of Vars needed from their relation; in that case, they

Re: [HACKERS] PostgreSQL Audit Extension

On 2/19/16 10:14 AM, Bruce Momjian wrote: > On Fri, Feb 19, 2016 at 09:19:58AM -0500, David Steele wrote: >>> I was suggesting we could track write events via logical replication and >>> then we only have to figure out auditing of read events, which would be >>> easier. >> >> I agree that DDL/DML a

Re: [HACKERS] PostgreSQL Audit Extension

On Fri, Feb 19, 2016 at 09:19:58AM -0500, David Steele wrote: > > I was suggesting we could track write events via logical replication and > > then we only have to figure out auditing of read events, which would be > > easier. > > I agree that DDL/DML audit logging requires a lot of the same > inf

Re: [HACKERS] pg_ctl promote wait

On Thu, Feb 18, 2016 at 5:45 PM, Simon Riggs wrote: > On 18 February 2016 at 08:33, Andres Freund wrote: >> >> Hi, >> >> On 2016-02-17 21:47:50 -0500, Peter Eisentraut wrote: >> > It would be nice if pg_ctl promote supported the -w (wait) option. +1 >> > How could pg_ctl determine when the prom

Re: [HACKERS] checkpointer continuous flushing - V16

Hello. Based on these results I think 32 will be a good default for checkpoint_flush_after? There's a few cases where 64 showed to be beneficial, and some where 32 is better. I've seen 64 perform a bit better in some cases here, but the differences were not too big. Yes, these many runs show

Re: [HACKERS] Relaxing SSL key permission checks

Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Now, I have heard it argued that the OpenSSH/L authors are a bunch of > idiots who know nothing about security. But it's not like insisting > on restrictive permissions on key files is something we invented out > of the blue. It's pretty standard pra

Re: [HACKERS] [PATCH v5] GSSAPI encryption support

On 2/15/16 12:45 PM, Robbie Harwood wrote: > David Steele writes: > >> 1) It didn't apply cleanly to HEAD. It did apply cleanly on a455878 >> which I figured was recent enough for testing. I didn't bisect to find >> the exact commit that broke it. > > It applied to head of master (57c932475504d

Re: [HACKERS] PostgreSQL Audit Extension

On 2/17/16 10:25 PM, Bruce Momjian wrote: > On Wed, Feb 17, 2016 at 01:59:09PM +0530, Robert Haas wrote: >> On Wed, Feb 17, 2016 at 5:20 AM, Bruce Momjian wrote: >>> On Fri, Feb 5, 2016 at 01:16:25PM -0500, Stephen Frost wrote: Looking at pgaudit and the other approaches to auditing which ha

Re: [HACKERS] MinGW / Windows / printf format specifiers

Craig Ringer writes: > On 19 February 2016 at 12:15, Chapman Flack wrote: >> Have issues like this been dealt with in PostgreSQL code before, and did >> a favorite approach emerge? > INT64_FORMAT and UINT64_FORMAT Yeah. Note in particular the convention to avoid using those in translatable str

Re: [HACKERS] Typo in bufmgr.c that result in waste of memory

Hi, Nice catch! On February 19, 2016 2:42:08 PM GMT+01:00, Tom Lane wrote: >> I think we should fix it, but not backpatch. > >I don't think that's particularly good policy. It's a clear bug, why >would we not fix it? Leaving it as-is in the back branches can have >no good effect, and what it d

Re: [HACKERS] Typo in bufmgr.c that result in waste of memory

Simon Riggs writes: > I see the problem, but I don't buy the argument that it wastes large > amounts of memory. Or do you have some evidence that it does? Agreed, it seems unlikely that that hash table gets large enough for this to be really significant. Still ... > I think we should fix it, bu

Re: [HACKERS] Re: BUG #13685: Archiving while idle every archive_timeout with wal_level hot_standby

On Fri, Feb 12, 2016 at 6:57 PM, Michael Paquier wrote: > > > OK, here is attached a new version that I hope addresses all the > points raised until now. The following things are changed: > - Extend XLogInsert with a new uint8 argument to have flags. As of now > there is one flag: XLOG_INSERT_NO_P

Re: [HACKERS] WAL logging problem in 9.4.3?

On Fri, Feb 19, 2016 at 4:33 PM, Michael Paquier wrote: > On Thu, Feb 18, 2016 at 4:27 PM, Michael Paquier > wrote: >> On Thu, Feb 4, 2016 at 3:24 PM, Heikki Linnakangas wrote: >>> I dropped the ball on this one back in July, so here's an attempt to revive >>> this thread. >>> >>> I spent some t

Re: [HACKERS] checkpointer continuous flushing - V16

Hi, On 2016-02-19 10:16:41 +0100, Fabien COELHO wrote: > Below the results of a lot of tests with pgbench to exercise checkpoints on > the above version when fetched. Wow, that's a great test series. > Overall comments: > - sorting & flushing is basically always a winner > - benchmarking with

Re: [HACKERS] Relaxing SSL key permission checks

Re: Tom Lane 2016-02-18 <27423.1455809...@sss.pgh.pa.us> > I did have a thought though: could we allow two distinct permissions > configurations? That is, allow either: > > * file is owned by us, mode 0600 or less > > * file is owned by root, mode 0640 or less > > The first case is what we allo

Re: [HACKERS] proposal: function parse_ident

Hi 2016-02-18 4:59 GMT+01:00 Pavel Stehule : > select >> parse_ident(E'X\rXX'); > > I am sending updated patch - I used json function for correct escaping - the escaping behave is same. Regards Pavel diff --git a/doc/src/sgml/func.sgml b/doc/src/s

Re: [HACKERS] Typo in bufmgr.c that result in waste of memory

> I see the problem, but I don't buy the argument that it wastes large amounts > of memory. Or do you have some evidence that it does? No. I don’t have any trouble caused by it. I think I did not mention it wastes 'large' amount of memory but 'a few'. > I think we should fix it, but not backpatch

Re: [HACKERS] plpgsql - DECLARE - cannot to use %TYPE or %ROWTYPE for composite types

It seems all fixes are done. I tested the patch and regression tests passed. On 27.01.2016 20:58, Pavel Stehule wrote: > --- 1681,1687 >* -- >*/ > PLpgSQL_type * > ! plpgsql_parse_wordtype(char *ident, int reftype_mode) > { > PL

Re: [HACKERS] Bug in StartupSUBTRANS

On 14 February 2016 at 00:03, Jeff Janes wrote: > I've attached a new version, incorporating comments from Tom and Michael. > Applied, thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & S

Re: [HACKERS] Typo in bufmgr.c that result in waste of memory

On 19 February 2016 at 02:58, Takashi Horikawa wrote: > I have just found a typo in the source code (not in a comment) of bufmgr.c > that result in waste of memory. It might be a 'bug' but it does not result > in any incorrect operation but just results in waste of a few memory > resource. > > A

Re: [HACKERS] checkpointer continuous flushing - V16

Hello Andres, I don't want to post a full series right now, but my working state is available on http://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git;a=shortlog;h=refs/heads/checkpoint-flush git://git.postgresql.org/git/users/andresfreund/postgres.git checkpoint-flush Below the

[PoC] WaitLatchOrSocketMulti (Re: [HACKERS] Performance degradation in commit ac1d794)

Hello, I don't see how ac1d794 will be dealt, but I tried an example implement of multi-socket version of WaitLatchOrSocket using callbacks on top of the current master where ac1d794 has not been removed yet. At Thu, 14 Jan 2016 13:46:44 -0500, Robert Haas wrote in > On Thu, Jan 14, 2016 at 12: