Re: [HACKERS] Declarative partitioning

On Sat, Feb 20, 2016 at 1:41 PM, Peter Eisentraut wrote: > On 2/16/16 9:56 PM, Amit Langote wrote: >> From now on, instead of attaching multiple files like in the previous >> message, I will send a single tar.gz which will contain patches created by >> git-format-patch. > >

Re: [HACKERS] [JDBC] JDBC behaviour

Hi All I understand your point, may be I didn't understand everyone or everyone didn't understand me one feature of PostgreSQL is implemented into another feature of Java ( i say subject PostgreSQL::autocommit Vs JDBC::setAutoCommit ), i.e PostgreSQL::"set autocommit to FALSE" is implemented as

Re: [HACKERS] Declarative partitioning

On 2/16/16 9:56 PM, Amit Langote wrote: > From now on, instead of attaching multiple files like in the previous > message, I will send a single tar.gz which will contain patches created by > git-format-patch. Please don't do that. -- Sent via pgsql-hackers mailing list

[HACKERS] GCC 6 warning fixes

Here are three patches to fix new warnings in GCC 6. 0001 is apparently a typo. 0002 was just (my?) stupid code to begin with. 0003 is more of a workaround. There could be other ways address this, too. From 1e5bf0bdcd86b807d881ea82245275389083ec75 Mon Sep 17 00:00:00 2001 From: Peter

Re: [HACKERS] exposing pg_controldata and pg_config as functions

On 01/17/2016 04:10 PM, Joe Conway wrote: > On 01/16/2016 06:02 AM, Michael Paquier wrote: >> On Wed, Dec 30, 2015 at 9:08 AM, Joe Conway wrote: >>> 3) Adds new functions, more or less in line with previous discussions: >>>* pg_checkpoint_state() >>>*

[HACKERS] psql metaqueries with \gexec

Often, I'm faced with a long .sql script that builds some objects, then builds things on top of them. This means that some of the queries I wish to run are dependent on the state of things that are unknown at the time of writing the script. I could give up, and make a python script that mostly

Re: [HACKERS] checkpointer continuous flushing - V16

On Sat, Feb 20, 2016 at 5:08 AM, Fabien COELHO wrote: >> Kernel 3.2 is extremely bad for Postgresql, as the vm seems to amplify IO >> somehow. The difference to 3.13 (the latest LTS kernel for 12.04) is huge. >> >> >>

Re: [HACKERS] checkpointer continuous flushing - V18

On 2016-02-19 22:46:44 +0100, Fabien COELHO wrote: > > Hello Andres, > > >Here's the next two (the most important) patches of the series: > >0001: Allow to trigger kernel writeback after a configurable number of > >writes. > >0002: Checkpoint sorting and balancing. > > I will look into these

Re: [HACKERS] checkpointer continuous flushing - V18

Hello Andres, Here's the next two (the most important) patches of the series: 0001: Allow to trigger kernel writeback after a configurable number of writes. 0002: Checkpoint sorting and balancing. I will look into these two in depth. Note that I would have ordered them in reverse because

Re: [HACKERS] proposal: make NOTIFY list de-duplication optional

On 2/9/16, Tom Lane wrote: > FWIW, I think it would be a good thing if the NOTIFY statement syntax were > not remarkably different from the syntax used in the pg_notify() function > call. To do otherwise would certainly be confusing. So on the whole > I'd go with the "NOTIFY

Re: [HACKERS] proposal: PL/Pythonu - function ereport

2016-02-18 17:59 GMT+01:00 Catalin Iacob : > On 2/18/16, Pavel Stehule wrote: > > it doesn't look badly. Is there any possibility how to emulate it with > > Python2 ? What do you think about some similar implementation on Python2? > > The example

Re: [HACKERS] pg_ctl promote wait

Andres Freund writes: > On 2016-02-19 15:09:58 -0500, Tom Lane wrote: >> I see no need for an additional mechanism. Just watch pg_control until >> you see DB_IN_PRODUCTION state there, then switch over to the same >> connection probing that "pg_ctl start -w" uses. > That's

Re: [HACKERS] pg_ctl promote wait

On 2016-02-19 15:09:58 -0500, Tom Lane wrote: > Peter Eisentraut writes: > > On 2/19/16 10:06 AM, Fujii Masao wrote: > >> One concern is that there can be a "time" after the pg_control's state > >> is changed to DB_IN_PRODUCTION and before the server is able to > >> start

Re: [HACKERS] checkpointer continuous flushing - V18

On 2016-02-04 16:54:58 +0100, Andres Freund wrote: > Hi, > > Fabien asked me to post a new version of the checkpoint flushing patch > series. While this isn't entirely ready for commit, I think we're > getting closer. > > I don't want to post a full series right now, but my working state is >

Re: [HACKERS] FDW: should GetFdwRoutine be called when drop table?

Andres Freund writes: > On 2016-02-19 14:18:19 -0500, Peter Eisentraut wrote: >> On 2/19/16 12:21 PM, Feng Tian wrote: >>> I have an fdw that each foreign table will acquire some persisted resource. >> But foreign data wrappers are meant to be wrappers around data managed >>

Re: [HACKERS] Declarative partitioning

On Thu, Feb 18, 2016 at 12:41 AM, Amit Langote < langote_amit...@lab.ntt.co.jp> wrote: > START [ EXCL ] (startval) END [ INCL ] (endval) > > That is, in range type notation, '[startval, endval)' is the default > behavior. So for each partition, there is at least the following pieces of >

Re: [HACKERS] pg_ctl promote wait

Peter Eisentraut writes: > On 2/19/16 10:06 AM, Fujii Masao wrote: >> One concern is that there can be a "time" after the pg_control's state >> is changed to DB_IN_PRODUCTION and before the server is able to >> start accepting normal (not read-only) connections. So if users try

Re: [HACKERS] checkpointer continuous flushing - V16

Hallo Patric, Kernel 3.2 is extremely bad for Postgresql, as the vm seems to amplify IO somehow. The difference to 3.13 (the latest LTS kernel for 12.04) is huge.

Re: [HACKERS] pg_ctl promote wait

Peter Eisentraut writes: > Is it safe to read pg_control externally without a lock? pg_controldata > will just report a CRC error and proceed, and if you're not sure you can > just run it again. But if a promotion fails every so often because of > concurrent pg_control writes,

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

On Fri, Feb 19, 2016 at 12:05 PM, Alvaro Herrera wrote: > But those pages are supposed to be used as the index grows. So unless > they are forgotten by the FSM, they shouldn't accumulate. (Except where > the table doesn't grow but only shrinks, so there's no need for

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

Peter Geoghegan wrote: > On Thu, Feb 18, 2016 at 4:53 PM, Tom Lane wrote: > >> there are usage patterns where half-dead pages might accumulate. > > > > Other than a usage pattern of "randomly SIGKILL backends every few > > seconds", I don't see how that would happen. > > I

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

On Thu, Feb 18, 2016 at 4:53 PM, Tom Lane wrote: > Only a physical-order scan, ie vacuum, would visit a dead page > (ignoring transient corner cases like a page getting deleted while an > indexscan is in flight to it). So I think treating it as part of the > fragmentation

Re: [HACKERS] Restructuring Paths to allow per-Path targetlist info

Alvaro Herrera writes: >> So, the attached patch just bites the bullet and adds explicit output >> tlist information to struct Path. > Hmm, I wonder if this can be used to attack the problem here in a more > sensible manner: >

Re: [HACKERS] FDW: should GetFdwRoutine be called when drop table?

On 2016-02-19 14:18:19 -0500, Peter Eisentraut wrote: > On 2/19/16 12:21 PM, Feng Tian wrote: > > I have an fdw that each foreign table will acquire some persisted resource. > > In my case, some files in file system. To drop the table cleanly, I > > have written > > an object_access_hook that

Re: [HACKERS] pg_ctl promote wait

On 2016-02-20 00:06:09 +0900, Fujii Masao wrote: > One concern is that there can be a "time" after the pg_control's state > is changed to DB_IN_PRODUCTION and before the server is able to > start accepting normal (not read-only) connections. So if users try to > start write transaction just after

Re: [HACKERS] FDW: should GetFdwRoutine be called when drop table?

On 2/19/16 12:21 PM, Feng Tian wrote: > I have an fdw that each foreign table will acquire some persisted resource. > In my case, some files in file system. To drop the table cleanly, I > have written > an object_access_hook that remove those files. The hook is installed in > _PG_init. > >

Re: [HACKERS] pg_ctl promote wait

On 2016-02-19 13:48:52 -0500, Peter Eisentraut wrote: > Is it safe to read pg_control externally without a lock? pg_controldata > will just report a CRC error and proceed, and if you're not sure you can > just run it again. But if a promotion fails every so often because of > concurrent

Re: [HACKERS] pg_ctl promote wait

On 2/19/16 10:06 AM, Fujii Masao wrote: > One concern is that there can be a "time" after the pg_control's state > is changed to DB_IN_PRODUCTION and before the server is able to > start accepting normal (not read-only) connections. So if users try to > start write transaction just after pg_ctl

Re: [HACKERS] pg_ctl promote wait

On 2/18/16 3:33 AM, Andres Freund wrote: > Hi, > > On 2016-02-17 21:47:50 -0500, Peter Eisentraut wrote: >> It would be nice if pg_ctl promote supported the -w (wait) option. >> >> How could pg_ctl determine when the promotion has finished? > > How about looking into pg_control?

Re: [HACKERS] about google summer of code 2016

On 02/19/2016 10:10 AM, Álvaro Hernández Tortosa wrote: Hi. Oleg and I discussed recently that a really good addition to a GSoC item would be to study whether it's convenient to have a binary serialization format for jsonb over the wire. Some argue this should be benchmarked first.

Re: [HACKERS] checkpointer continuous flushing - V16

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Fabien, Fabien COELHO schrieb am 19.02.2016 um 16:04: > >>> [...] Ubuntu 12.04 LTS (precise) >> >> That's with 12.04's standard kernel? > > Yes. Kernel 3.2 is extremely bad for Postgresql, as the vm seems to amplify IO somehow. The difference

Re: [HACKERS] about google summer of code 2016

Hi. Oleg and I discussed recently that a really good addition to a GSoC item would be to study whether it's convenient to have a binary serialization format for jsonb over the wire. Some argue this should be benchmarked first. So the scope for this project would be to benchmark and

[HACKERS] Google Summer of Code

Mentoring organization application deadline is 19 February 19:00 UTC

[HACKERS] FDW: should GetFdwRoutine be called when drop table?

Hi, Hackers, I have an fdw that each foreign table will acquire some persisted resource. In my case, some files in file system. To drop the table cleanly, I have written an object_access_hook that remove those files. The hook is installed in _PG_init. It all worked well except one case.

Re: [HACKERS] PostgreSQL Audit Extension

On Fri, Feb 19, 2016 at 08:20:31AM -0800, Andres Freund wrote: > On 2016-02-19 10:14:47 -0500, Bruce Momjian wrote: > > We have already hesitated to record DDL changes for > > logical replication because of the code size, maintenance overhead, and > > testing required. > > I'm not sure what

Re: [HACKERS] PostgreSQL Audit Extension

On Fri, Feb 19, 2016 at 11:20:13AM -0500, David Steele wrote: > On 2/19/16 10:54 AM, Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > >> Understood. My point is that there is a short list of read events, and > >> many DDL events. We have already hesitated to record DDL changes for > >>

Re: [HACKERS] about google summer of code 2016

Atri Sharma wrote: > I agree, there might be scope for non core projects and PL/Java sounds like > a good area. We've hosted MADlib-based projects in the past, so why not. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training &

Re: [HACKERS] PostgreSQL Audit Extension

On 2016-02-19 10:14:47 -0500, Bruce Momjian wrote: > We have already hesitated to record DDL changes for > logical replication because of the code size, maintenance overhead, and > testing required. I'm not sure what you're referring to here? It'd be some relatively minor code surgery to also

Re: [HACKERS] PostgreSQL Audit Extension

On 2/19/16 10:54 AM, Alvaro Herrera wrote: > Bruce Momjian wrote: > >> Understood. My point is that there is a short list of read events, and >> many DDL events. We have already hesitated to record DDL changes for >> logical replication because of the code size, maintenance overhead, and >>

Re: [HACKERS] PostgreSQL Audit Extension

On Fri, Feb 19, 2016 at 12:54:17PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > Understood. My point is that there is a short list of read events, and > > many DDL events. We have already hesitated to record DDL changes for > > logical replication because of the code size,

Re: [HACKERS] PostgreSQL Audit Extension

Bruce Momjian wrote: > Understood. My point is that there is a short list of read events, and > many DDL events. We have already hesitated to record DDL changes for > logical replication because of the code size, maintenance overhead, and > testing required. DDL is already captured using the

Re: [HACKERS] Restructuring Paths to allow per-Path targetlist info

Tom Lane wrote: > So, the attached patch just bites the bullet and adds explicit output > tlist information to struct Path. I did set things up so that the cost > is only one pointer in each Path in the typical case where Paths emit > the set of Vars needed from their relation; in that case,

Re: [HACKERS] PostgreSQL Audit Extension

On 2/19/16 10:14 AM, Bruce Momjian wrote: > On Fri, Feb 19, 2016 at 09:19:58AM -0500, David Steele wrote: >>> I was suggesting we could track write events via logical replication and >>> then we only have to figure out auditing of read events, which would be >>> easier. >> >> I agree that DDL/DML

Re: [HACKERS] PostgreSQL Audit Extension

On Fri, Feb 19, 2016 at 09:19:58AM -0500, David Steele wrote: > > I was suggesting we could track write events via logical replication and > > then we only have to figure out auditing of read events, which would be > > easier. > > I agree that DDL/DML audit logging requires a lot of the same >

Re: [HACKERS] pg_ctl promote wait

On Thu, Feb 18, 2016 at 5:45 PM, Simon Riggs wrote: > On 18 February 2016 at 08:33, Andres Freund wrote: >> >> Hi, >> >> On 2016-02-17 21:47:50 -0500, Peter Eisentraut wrote: >> > It would be nice if pg_ctl promote supported the -w (wait) option. +1

Re: [HACKERS] checkpointer continuous flushing - V16

Hello. Based on these results I think 32 will be a good default for checkpoint_flush_after? There's a few cases where 64 showed to be beneficial, and some where 32 is better. I've seen 64 perform a bit better in some cases here, but the differences were not too big. Yes, these many runs show

Re: [HACKERS] Relaxing SSL key permission checks

Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Now, I have heard it argued that the OpenSSH/L authors are a bunch of > idiots who know nothing about security. But it's not like insisting > on restrictive permissions on key files is something we invented out > of the blue. It's pretty standard

Re: [HACKERS] [PATCH v5] GSSAPI encryption support

On 2/15/16 12:45 PM, Robbie Harwood wrote: > David Steele writes: > >> 1) It didn't apply cleanly to HEAD. It did apply cleanly on a455878 >> which I figured was recent enough for testing. I didn't bisect to find >> the exact commit that broke it. > > It applied to head of

Re: [HACKERS] PostgreSQL Audit Extension

On 2/17/16 10:25 PM, Bruce Momjian wrote: > On Wed, Feb 17, 2016 at 01:59:09PM +0530, Robert Haas wrote: >> On Wed, Feb 17, 2016 at 5:20 AM, Bruce Momjian wrote: >>> On Fri, Feb 5, 2016 at 01:16:25PM -0500, Stephen Frost wrote: Looking at pgaudit and the other approaches

Re: [HACKERS] MinGW / Windows / printf format specifiers

Craig Ringer writes: > On 19 February 2016 at 12:15, Chapman Flack wrote: >> Have issues like this been dealt with in PostgreSQL code before, and did >> a favorite approach emerge? > INT64_FORMAT and UINT64_FORMAT Yeah. Note in particular the

Re: [HACKERS] Typo in bufmgr.c that result in waste of memory

Hi, Nice catch! On February 19, 2016 2:42:08 PM GMT+01:00, Tom Lane wrote: >> I think we should fix it, but not backpatch. > >I don't think that's particularly good policy. It's a clear bug, why >would we not fix it? Leaving it as-is in the back branches can have >no good

Re: [HACKERS] Typo in bufmgr.c that result in waste of memory

Simon Riggs writes: > I see the problem, but I don't buy the argument that it wastes large > amounts of memory. Or do you have some evidence that it does? Agreed, it seems unlikely that that hash table gets large enough for this to be really significant. Still ... > I

Re: [HACKERS] Re: BUG #13685: Archiving while idle every archive_timeout with wal_level hot_standby

On Fri, Feb 12, 2016 at 6:57 PM, Michael Paquier wrote: > > > OK, here is attached a new version that I hope addresses all the > points raised until now. The following things are changed: > - Extend XLogInsert with a new uint8 argument to have flags. As of now > there

Re: [HACKERS] WAL logging problem in 9.4.3?

On Fri, Feb 19, 2016 at 4:33 PM, Michael Paquier wrote: > On Thu, Feb 18, 2016 at 4:27 PM, Michael Paquier > wrote: >> On Thu, Feb 4, 2016 at 3:24 PM, Heikki Linnakangas wrote: >>> I dropped the ball on this one back in

Re: [HACKERS] checkpointer continuous flushing - V16

Hi, On 2016-02-19 10:16:41 +0100, Fabien COELHO wrote: > Below the results of a lot of tests with pgbench to exercise checkpoints on > the above version when fetched. Wow, that's a great test series. > Overall comments: > - sorting & flushing is basically always a winner > - benchmarking

Re: [HACKERS] Relaxing SSL key permission checks

Re: Tom Lane 2016-02-18 <27423.1455809...@sss.pgh.pa.us> > I did have a thought though: could we allow two distinct permissions > configurations? That is, allow either: > > * file is owned by us, mode 0600 or less > > * file is owned by root, mode 0640 or less > > The first case is what we

Re: [HACKERS] proposal: function parse_ident

Hi 2016-02-18 4:59 GMT+01:00 Pavel Stehule : > select >> parse_ident(E'X\rXX'); > > I am sending updated patch - I used json function for correct escaping - the escaping behave is same. Regards Pavel diff --git

Re: [HACKERS] Typo in bufmgr.c that result in waste of memory

> I see the problem, but I don't buy the argument that it wastes large amounts > of memory. Or do you have some evidence that it does? No. I don’t have any trouble caused by it. I think I did not mention it wastes 'large' amount of memory but 'a few'. > I think we should fix it, but not

Re: [HACKERS] plpgsql - DECLARE - cannot to use %TYPE or %ROWTYPE for composite types

It seems all fixes are done. I tested the patch and regression tests passed. On 27.01.2016 20:58, Pavel Stehule wrote: > --- 1681,1687 >* -- >*/ > PLpgSQL_type * > ! plpgsql_parse_wordtype(char *ident, int reftype_mode) > { >

Re: [HACKERS] Bug in StartupSUBTRANS

On 14 February 2016 at 00:03, Jeff Janes wrote: > I've attached a new version, incorporating comments from Tom and Michael. > Applied, thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support,

Re: [HACKERS] Typo in bufmgr.c that result in waste of memory

On 19 February 2016 at 02:58, Takashi Horikawa wrote: > I have just found a typo in the source code (not in a comment) of bufmgr.c > that result in waste of memory. It might be a 'bug' but it does not result > in any incorrect operation but just results in waste of a

Re: [HACKERS] checkpointer continuous flushing - V16

Hello Andres, I don't want to post a full series right now, but my working state is available on http://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git;a=shortlog;h=refs/heads/checkpoint-flush git://git.postgresql.org/git/users/andresfreund/postgres.git checkpoint-flush Below

[PoC] WaitLatchOrSocketMulti (Re: [HACKERS] Performance degradation in commit ac1d794)

Hello, I don't see how ac1d794 will be dealt, but I tried an example implement of multi-socket version of WaitLatchOrSocket using callbacks on top of the current master where ac1d794 has not been removed yet. At Thu, 14 Jan 2016 13:46:44 -0500, Robert Haas wrote in