Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Amit Langote
Fabien, On 2015-06-03 PM 02:53, Fabien COELHO wrote: > >> >> It seems 'sync' gets closer to what I really wanted 'flush' to mean. If I >> understand this and the previous discussion(s) correctly, the patch tries to >> alleviate the problems caused by one-big-sync-at-the end-of-writes by doing >>

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Fabien COELHO
I agree with you that if we have to add a sort phase, there is additional work and that work could be significant depending on the design we choose, however without that, this patch can have impact on many kind of workloads, even in your mail in one of the tests ("pgbench -M prepared -N -T 100 -

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Fabien COELHO
Hello Amit, It is currently "*_flush_to_disk". In Andres Freund version the name is "sync_on_checkpoint_flush", but I did not found it very clear. Using "*_flush_on_write" instead as your suggest, would be fine as well, it emphasizes the "when/how" it occurs instead of the final "destination",

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Fabien COELHO
That might be the case in a database with a single small table; i.e. where all the writes go to a single file. But as soon as you have large tables (i.e. many segments) or multiple tables, a significant part of the writes issued independently from checkpointing will be outside the processing

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Amit Kapila
On Tue, Jun 2, 2015 at 6:45 PM, Fabien COELHO wrote: > > > Hello Amit, > >> [...] >>> >>> The objective is to help avoid PG stalling when fsyncing on checkpoints, >>> and in general to get better latency-bound performance. >> >> >> Won't this lead to more-unsorted writes (random I/O) as the >> Flu

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-02 Thread Amit Kapila
On Tue, Jun 2, 2015 at 10:26 PM, Andrew Dunstan wrote: > > On 05/15/2015 02:21 AM, Amit Kapila wrote: >> >> >> Find the patch which gets rid of rmtree usage. I have made it as >> a separate function because the same code is used from >> create_tablespace_directories() as well. I thought of exte

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Alvaro Herrera
Thomas Munro wrote: > On Tue, Jun 2, 2015 at 9:30 AM, Alvaro Herrera > wrote: > > My guess is that the file existed, and perhaps had one or more pages, > > but the wanted page doesn't exist, so we tried to read but got 0 bytes > > back. read() returns 0 in this case but doesn't set errno. > > >

Re: [HACKERS] why does txid_current() assign new transaction-id?

2015-06-02 Thread Fujii Masao
On Wed, Jun 3, 2015 at 9:04 AM, Naoya Anzai wrote: >> > diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml >> > index 89a609f..6485b42 100644 >> > --- a/doc/src/sgml/func.sgml >> > +++ b/doc/src/sgml/func.sgml >> > @@ -16233,7 +16233,7 @@ SELECT collation for ('foo' COLLATE "de_DE"); >>

Re: [HACKERS] nested loop semijoin estimates

2015-06-02 Thread Tom Lane
Tomas Vondra writes: > On 06/02/15 23:27, Tom Lane wrote: >> Do we have instructions around here anyplace on how to set up/use >>> TPC-DS? I couldn't find anything about it on the wiki ... > Not that I'm aware of, but it's not really all that difficult. > [ instructions... ] Thanks. I added som

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Thomas Munro
On Tue, Jun 2, 2015 at 9:30 AM, Alvaro Herrera wrote: > My guess is that the file existed, and perhaps had one or more pages, > but the wanted page doesn't exist, so we tried to read but got 0 bytes > back. read() returns 0 in this case but doesn't set errno. > > I didn't find a way to set things

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Amit Langote
Hi, On 2015-06-02 PM 07:19, Fabien COELHO wrote: > >> Not that the GUC naming is the most pressing issue here, but do you think >> "*_flush_on_write" describes what the patch does? > > It is currently "*_flush_to_disk". In Andres Freund version the name is > "sync_on_checkpoint_flush", but I di

Re: [HACKERS] why does txid_current() assign new transaction-id?

2015-06-02 Thread Naoya Anzai
> > diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml > > index 89a609f..6485b42 100644 > > --- a/doc/src/sgml/func.sgml > > +++ b/doc/src/sgml/func.sgml > > @@ -16233,7 +16233,7 @@ SELECT collation for ('foo' COLLATE "de_DE"); > > > > > > txid_current() > > bigi

Re: [HACKERS] Minor issue with BRIN regression tests

2015-06-02 Thread Peter Geoghegan
On Mon, Jun 1, 2015 at 7:35 PM, Peter Geoghegan wrote: > Attached patch adjusts BRIN regression tests to make a non-obvious > dependency on tuple order explicit. Currently, an index-only scan plan > is used by the query that I've adjusted. I'd rather be sure that that > continues. Here is another

Re: [HACKERS] pg_xlog -> pg_xjournal?

2015-06-02 Thread David Steele
On 5/31/15 1:46 PM, Tom Lane wrote: > Hm. I think the impact on third-party backup tools would be rather bad, > but there's a simple modification of the idea that might fix that: > just always create pg_xlog as a symlink to pg_xjournal during initdb. > Anybody who blindly removes pg_xlog won't hav

Re: [HACKERS] nested loop semijoin estimates

2015-06-02 Thread Tom Lane
I wrote: > Hm. In principle, add_path_precheck shouldn't be doing anything except > rejecting paths that would get rejected anyway later on. However it > strikes me that there's a logic error in it; specifically, this > simplifying assumption is wrong: > * For speed, we make exact rather

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Andres Freund
> > Hm. If GetOldestMultiXactOnDisk() gets the starting point by scanning > > the disk it'll always get one at a segment boundary, right? I'm not sure > > that's actually ok; because the value at the beginning of the segment > > can very well end up being a 0, as MaybeExtendOffsetSlru() will have >

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Robert Haas
On Tue, Jun 2, 2015 at 4:19 PM, Andres Freund wrote: > I'm not really convinced tying things closer to having done trimming is > easier to understand than tying things to recovery having finished. > > E.g. > if (did_trim) > oldestOffset = GetOldestReferencedOffset(oldest_da

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Andres Freund
On 2015-06-01 14:22:32 -0400, Robert Haas wrote: > commit d33b4eb0167f465edb00bd6c0e1bcaa67dd69fe9 > Author: Robert Haas > Date: Fri May 29 14:35:53 2015 -0400 > > foo Hehe! > diff --git a/src/backend/access/transam/multixact.c > b/src/backend/access/transam/multixact.c > index 9568ff1.

Re: [HACKERS] nested loop semijoin estimates

2015-06-02 Thread Tom Lane
Tomas Vondra writes: > So, if required_outer=false and both p->_startup=false, we get > consider_startup=false irrespectedly of the required_outer value, so > (!consider_startupe) != required_outer > so that the conditions are not equivalent. And indeed, by reverting the > if condition to t

Re: [HACKERS] nested loop semijoin estimates

2015-06-02 Thread Tomas Vondra
On 06/02/15 21:39, Tom Lane wrote: I wrote: I'm a bit disturbed by that, because AFAICS from the plans, these queries did not involve any semi or anti joins, which should mean that the patch would not have changed the planner's behavior. After contemplating my navel for awhile, it occurred t

Re: [HACKERS] nested loop semijoin estimates

2015-06-02 Thread Tom Lane
I wrote: > I'm a bit disturbed by that, because AFAICS from the plans, these queries > did not involve any semi or anti joins, which should mean that the patch > would not have changed the planner's behavior. After contemplating my navel for awhile, it occurred to me that maybe the patch's changes

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Andres Freund
On 2015-06-02 18:59:05 +0200, Fabien COELHO wrote: > > >>>IMO this feature, if done correctly, should result in better performance > >>>in 95+% of the workloads > >> > >>To demonstrate that would require time... > > > >Well, that's part of the contribution process. Obviously you can't test > >100%

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Andres Freund
On 2015-06-02 11:49:56 -0400, Robert Haas wrote: > On Tue, Jun 2, 2015 at 11:44 AM, Andres Freund wrote: > > On 2015-06-02 11:37:02 -0400, Robert Haas wrote: > >> The exact circumstances under which we're willing to replace a > >> relminmxid with a newly-computed one that differs are not altogethe

Re: [HACKERS] pg_xlog -> pg_xjournal?

2015-06-02 Thread Christian Ullrich
* Michael Nolan wrote: > Why not take a simpler approach and create a zero length file in directories > that should not be fiddled with by non-experts using a file name something > like "DO.NOT.DELETE.THESE.FILES"? Move the critical things into a new subdirectory? $PGDATA/pg_critical/pg_xlog?

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Fabien COELHO
IMO this feature, if done correctly, should result in better performance in 95+% of the workloads To demonstrate that would require time... Well, that's part of the contribution process. Obviously you can't test 100% of the problems, but you can work hard with coming up with very adversarial

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-02 Thread Andrew Dunstan
On 05/15/2015 02:21 AM, Amit Kapila wrote: On Thu, May 14, 2015 at 10:29 PM, Andrew Dunstan > wrote: > > > On 05/14/2015 10:52 AM, Robert Haas wrote: >> >> On Thu, May 14, 2015 at 12:12 AM, Amit Kapila mailto:amit.kapil...@gmail.com>> wrote: >>> >>> On Thu, May 14,

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-02 Thread David Fetter
On Mon, Jun 01, 2015 at 05:48:16PM -0300, Alvaro Herrera wrote: > Peter Eisentraut wrote: > > > But it is a valid point that we'd need to build up more extension > > API tests before we start cutting back significantly on the > > maybe-example-maybe-real extensions that we ship in contrib. We > >

Re: [HACKERS] Missing "-i / --ignore-version" in pg_dump help

2015-06-02 Thread Fujii Masao
On Fri, May 22, 2015 at 11:01 PM, Tom Lane wrote: > Fujii Masao writes: >> On Fri, May 22, 2015 at 8:59 PM, Fabrízio de Royes Mello >> wrote: >>> There are some reason to "-i, --ignore-version" option doesn't appear in >>> pg_dump help? > >> Because it's obsolete option, I think. > > Yeah, see c

Re: [HACKERS] [PATCH] Add error handling to byteaout.

2015-06-02 Thread Andreas Seltenreich
Tom Lane writes: > Andreas Seltenreich writes: >> The scary one is due to an integer overflow the attached patch also >> fixes. > > s/int/Size/ doesn't fix anything on 32-bit machines. Well, it changes the signedness of the computation on 32-bit, and in combination with the fact that "len" is a

Re: [HACKERS] WIP: Enhanced ALTER OPERATOR

2015-06-02 Thread Robert Haas
On Fri, May 29, 2015 at 4:28 AM, Alexander Korotkov wrote: > On Thu, May 28, 2015 at 6:43 PM, Tom Lane wrote: >> >> Alexander Korotkov writes: >> > Could we address both this problems by denying changing existing >> > commutators and negator? ISTM that setting absent commutator and negator >> >

[HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Noah Misch
On Tue, Jun 02, 2015 at 11:16:22AM -0400, Robert Haas wrote: > On Tue, Jun 2, 2015 at 1:21 AM, Noah Misch wrote: > > On Mon, Jun 01, 2015 at 02:06:05PM -0400, Robert Haas wrote: > > Granted. Would it be better to update both functions at the same time, and > > perhaps to make that a master-only

Re: [HACKERS] nested loop semijoin estimates

2015-06-02 Thread Tom Lane
Tomas Vondra writes: > On 06/02/15 16:37, Tom Lane wrote: >> It's possible that the change was due to random variation in ANALYZE >> statistics, in which case it was just luck. > I don't think so. I simply loaded the data, ran ANALYZE, and then simply > started either master or patched master. T

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Robert Haas
On Tue, Jun 2, 2015 at 11:44 AM, Andres Freund wrote: > On 2015-06-02 11:37:02 -0400, Robert Haas wrote: >> The exact circumstances under which we're willing to replace a >> relminmxid with a newly-computed one that differs are not altogether >> clear to me, but there's an "if" statement protectin

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Robert Haas
On Tue, Jun 2, 2015 at 11:36 AM, Andres Freund wrote: >> That would be a departure from the behavior of every existing release >> that includes this code based on, to my knowledge, zero trouble >> reports. > > On the other hand we're now at about bug #5 attributeable to the odd way > truncation wo

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Andres Freund
On 2015-06-02 11:37:02 -0400, Robert Haas wrote: > The exact circumstances under which we're willing to replace a > relminmxid with a newly-computed one that differs are not altogether > clear to me, but there's an "if" statement protecting that logic, so > there are some circumstances in which we'

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Robert Haas
On Tue, Jun 2, 2015 at 11:27 AM, Andres Freund wrote: > On 2015-06-02 11:16:22 -0400, Robert Haas wrote: >> I'm having trouble figuring out what to do about this. I mean, the >> essential principle of this patch is that if we can't count on >> relminmxid, datminmxid, or the control file to be acc

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Andres Freund
On 2015-06-02 11:29:24 -0400, Robert Haas wrote: > On Tue, Jun 2, 2015 at 8:56 AM, Andres Freund wrote: > > But what *definitely* looks wrong to me is that a TruncateMultiXact() in > > this scenario now (since a couple weeks ago) does a > > SimpleLruReadPage_ReadOnly() in the members slru via > >

Re: [HACKERS] nested loop semijoin estimates

2015-06-02 Thread Tomas Vondra
On 06/02/15 16:37, Tom Lane wrote: Tomas Vondra writes: OK, so I did the testing today - with TPC-H and TPC-DS benchmarks. The results are good, IMHO. I'm a bit disturbed by that, because AFAICS from the plans, these queries did not involve any semi or anti joins, which should mean that the

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Robert Haas
On Tue, Jun 2, 2015 at 8:56 AM, Andres Freund wrote: > But what *definitely* looks wrong to me is that a TruncateMultiXact() in > this scenario now (since a couple weeks ago) does a > SimpleLruReadPage_ReadOnly() in the members slru via > find_multixact_start(). That just won't work acceptably whe

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Andres Freund
On 2015-06-02 11:16:22 -0400, Robert Haas wrote: > I'm having trouble figuring out what to do about this. I mean, the > essential principle of this patch is that if we can't count on > relminmxid, datminmxid, or the control file to be accurate, we can at > least look at what is present on the disk

[HACKERS] [PATCH] Fix documentation bug in how to calculate the quasi-unique pg_log session_id

2015-06-02 Thread Joel Jacobson
Fix documentation bug in how to calculate the quasi-unique pg_log session_id While the code is truncating the backend_start time, the query example in the documentation is rouding. Fix by doing the same thing in the documentation as in, i.e. truncating the backend_start. elog.c: snprintf(strfbuf,

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Andres Freund
On 2015-06-02 17:01:50 +0200, Fabien COELHO wrote: > >The actual problem is sorting & fsyncing in a way that deals efficiently > >with tablespaces, i.e. doesn't write to tablespaces one-by-one. > >Not impossible, but it requires some thought. > > Hmmm... I would have neglected this point in a firs

[HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Robert Haas
On Tue, Jun 2, 2015 at 1:21 AM, Noah Misch wrote: > On Mon, Jun 01, 2015 at 02:06:05PM -0400, Robert Haas wrote: >> On Mon, Jun 1, 2015 at 12:46 AM, Noah Misch wrote: >> > On Fri, May 29, 2015 at 03:08:11PM -0400, Robert Haas wrote: >> >> SetMultiXactIdLimit() bracketed certain parts of its >> >>

Re: [HACKERS] [PATCH] Add error handling to byteaout.

2015-06-02 Thread Tom Lane
Andreas Seltenreich writes: > The scary one is due to an integer overflow the attached patch also > fixes. s/int/Size/ doesn't fix anything on 32-bit machines. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to yo

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Fabien COELHO
Hmmm. I think it should be implemented as Tom suggested, that is per chunks of shared buffers, in order to avoid allocating a "large" memory. I don't necessarily agree. But that's really just a minor implementation detail. Probably. The actual problem is sorting & fsyncing in a way that dea

[HACKERS] [PATCH] Add error handling to byteaout.

2015-06-02 Thread Andreas Seltenreich
Hi, when dealing with bytea values that are below the 1GB limit, but too large to be escaped, the error messages emitted are not very helpful for users. Especially if they appear in an unrelated context such as during pg_dump. I've attached a patch that adds ereport()ing that would have prevente

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Andres Freund
On 2015-06-02 15:42:14 +0200, Fabien COELHO wrote: > >>This version seems already quite effective and very light. ISTM that > >>adding a sort phase would mean reworking significantly how the > >>checkpointer processes pages. > > > >Meh. The patch for that wasn't that big. > > Hmmm. I think it shou

Re: [HACKERS] pg_xlog -> pg_xjournal?

2015-06-02 Thread Amit Langote
On Tue, Jun 2, 2015 at 11:03 PM, Abhijit Menon-Sen wrote: > At 2015-06-01 23:35:23 -0500, htf...@gmail.com wrote: >> >> No, it won't prevent the incredibly stupid from doing incredibly >> stupid things, nothing will. > > I hate to speechify, but I think we should try hard to avoid framing > such q

Re: [HACKERS] nested loop semijoin estimates

2015-06-02 Thread Tom Lane
Tomas Vondra writes: > OK, so I did the testing today - with TPC-H and TPC-DS benchmarks. The > results are good, IMHO. > With TPC-H, I've used 1GB and 4GB datasets, and I've seen no plan > changes at all. I don't plan to run the tests on larger data sets, I do > expect the behavior to remain

Re: [HACKERS] pg_xlog -> pg_xjournal?

2015-06-02 Thread Bruce Momjian
On Tue, Jun 2, 2015 at 07:33:19PM +0530, Abhijit Menon-Sen wrote: > At 2015-06-01 23:35:23 -0500, htf...@gmail.com wrote: > > > > No, it won't prevent the incredibly stupid from doing incredibly > > stupid things, nothing will. > > I hate to speechify, but I think we should try hard to avoid fram

Re: [HACKERS] pg_xlog -> pg_xjournal?

2015-06-02 Thread Joshua D. Drake
On 06/01/2015 09:35 PM, Michael Nolan wrote: Why not take a simpler approach and create a zero length file in directories that should not be fiddled with by non-experts using a file name something like "DO.NOT.DELETE.THESE.FILES"? +1 No, it won't prevent the incredibly stupid from doing inc

Re: [HACKERS] nested loop semijoin estimates

2015-06-02 Thread Tomas Vondra
On 06/02/15 01:47, Josh Berkus wrote: On 06/01/2015 03:22 PM, Tomas Vondra wrote: On 06/01/15 23:47, Josh Berkus wrote: On 06/01/2015 02:18 PM, Tom Lane wrote: Anybody else want to speak for or against back-patching the patch as posted? I intentionally didn't push it in before today's relea

Re: [HACKERS] pg_xlog -> pg_xjournal?

2015-06-02 Thread Abhijit Menon-Sen
At 2015-06-01 23:35:23 -0500, htf...@gmail.com wrote: > > No, it won't prevent the incredibly stupid from doing incredibly > stupid things, nothing will. I hate to speechify, but I think we should try hard to avoid framing such questions in terms of "incredibly stupid" people and the things they m

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Fabien COELHO
Hello Andres, I would rather separate them, unless this is a blocker. I think it is a blocker. Hmmm. This is an argument... This version seems already quite effective and very light. ISTM that adding a sort phase would mean reworking significantly how the checkpointer processes pages.

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Andres Freund
On 2015-06-02 15:15:39 +0200, Fabien COELHO wrote: > >Won't this lead to more-unsorted writes (random I/O) as the > >FlushBuffer requests (by checkpointer or bgwriter) are not sorted as > >per files or order of blocks on disk? > > Yep, probably. Under "moderate load" this is not an issue. The io-s

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Fabien COELHO
Hello Amit, [...] The objective is to help avoid PG stalling when fsyncing on checkpoints, and in general to get better latency-bound performance. Won't this lead to more-unsorted writes (random I/O) as the FlushBuffer requests (by checkpointer or bgwriter) are not sorted as per files or ord

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Fabien COELHO
Hello Andres, If I see correctly you picked up the version without sorting durch checkpoints. I think that's not going to work - there'll be too many situations where the new behaviour will be detrimental. Did you consider combining both approaches? Ja, I thought that it was a more complex p

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Andres Freund
On 2015-06-01 14:22:32 -0400, Robert Haas wrote: > On Mon, Jun 1, 2015 at 4:58 AM, Andres Freund wrote: > > The lack of WAL logging actually has caused problems in the 9.3.3 (?) > > era, where we didn't do any truncation during recovery... > > Right, but now we're piggybacking on the checkpoint r

Re: [HACKERS] pg_xlog -> pg_xjournal?

2015-06-02 Thread Andrew Dunstan
On 06/02/2015 03:06 AM, Joel Jacobson wrote: On Tue, Jun 2, 2015 at 6:35 AM, Michael Nolan wrote: Why not take a simpler approach and create a zero length file in directories that should not be fiddled with by non-experts using a file name something like "DO.NOT.DELETE.THESE.FILES"? Then the

[HACKERS] Streaming replication for psycopg2

2015-06-02 Thread Shulgin, Oleksandr
Hello, I've submitted a patch to psycopg2 to support streaming replication protocol (COPY_BOTH): https://github.com/psycopg/psycopg2/pull/322 It would be great if more people had a chance to take a look and provide feedback about it. In particular, please see example usage at this github comment

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Amit Kapila
On Mon, Jun 1, 2015 at 5:10 PM, Fabien COELHO wrote: > > Hello pg-devs, > > This patch is a simplified and generalized version of Andres Freund's > August 2014 patch for flushing while writing during checkpoints, with some > documentation and configuration warnings added. > > For the initial patc

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Andres Freund
Hi, It's nice to see the topic being picked up. If I see correctly you picked up the version without sorting durch checkpoints. I think that's not going to work - there'll be too many situations where the new behaviour will be detrimental. Did you consider combining both approaches? Greetings,

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Fabien COELHO
Hello Amit, Not that the GUC naming is the most pressing issue here, but do you think "*_flush_on_write" describes what the patch does? It is currently "*_flush_to_disk". In Andres Freund version the name is "sync_on_checkpoint_flush", but I did not found it very clear. Using "*_flush_on_wr

Re: [HACKERS] pg_xlog -> pg_xjournal?

2015-06-02 Thread Mark Kirkwood
On 01/06/15 05:29, Joel Jacobson wrote: While anyone who is familiar with postgres would never do something as stupid as to delete pg_xlog, according to Google, there appears to be a fair amount of end-users out there having made the irrevocable mistake of deleting the precious directory, a decis

Re: [HACKERS] checkpointer continuous flushing

2015-06-02 Thread Amit Langote
Hi Fabien, On 2015-06-01 PM 08:40, Fabien COELHO wrote: > > Turning "checkpoint_flush_to_disk = on" reduces significantly the number > of late transactions. These late transactions are not uniformly distributed, > but are rather clustered around times when pg is stalled, i.e. more or less > unre

Re: [HACKERS] [GENERAL] psql weird behaviour with charset encodings

2015-06-02 Thread Michael Paquier
On Sun, May 24, 2015 at 2:43 AM, Noah Misch wrote: > On Sat, May 08, 2010 at 09:24:45PM -0400, Tom Lane wrote: >> hgonza...@gmail.com writes: >> > http://sources.redhat.com/bugzilla/show_bug.cgi?id=649 >> >> > The last explains why they do not consider it a bug: >> >> > ISO C99 requires for %.*s t

Re: [HACKERS] auto_explain sample rate

2015-06-02 Thread Pavel Stehule
2015-06-02 9:07 GMT+02:00 Craig Ringer : > On 29 May 2015 at 11:35, Tom Lane wrote: > >> Craig Ringer writes: >> > It's sometimes desirable to collect auto_explain data with ANALYZE in >> order >> > to track down hard-to-reproduce issues, but the performance impacts can >> be >> > pretty hefty o

Re: [HACKERS] auto_explain sample rate

2015-06-02 Thread Craig Ringer
On 29 May 2015 at 11:35, Tom Lane wrote: > Craig Ringer writes: > > It's sometimes desirable to collect auto_explain data with ANALYZE in > order > > to track down hard-to-reproduce issues, but the performance impacts can > be > > pretty hefty on the DB. > > > I'm inclined to add a sample rate t

Re: [HACKERS] pg_xlog -> pg_xjournal?

2015-06-02 Thread Joel Jacobson
On Tue, Jun 2, 2015 at 6:35 AM, Michael Nolan wrote: > Why not take a simpler approach and create a zero length file in directories > that should not be fiddled with by non-experts using a file name something > like "DO.NOT.DELETE.THESE.FILES"? Then the smart sysadmin will say "but I didn't delet

Re: [HACKERS] [NOVICE] psql readline Tab insert tab

2015-06-02 Thread Albe Laurenz
Peter Eisentraut wrote: > On 6/1/15 7:00 AM, Albe Laurenz wrote: >> Hans Ginzel wrote: > how to make psql (readline) to insert tab when Tab is pressed? E.g. for > pasting. I know, there is -n option. But then the history is not > accessible. It could be done by adding the followin