Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-09-11 Thread Michael Paquier
On Fri, Sep 12, 2014 at 3:19 PM, Michael Paquier wrote: > On Mon, Feb 3, 2014 at 3:42 PM, Amit Kapila wrote: >> On Wed, Jan 15, 2014 at 2:43 AM, Simon Riggs wrote: >>> On 8 January 2014 08:33, Simon Riggs wrote: >>> >>> Patch attached, implemented to reduce writes by SELECTs only. > > This patc

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Amit Kapila
On Thu, Sep 11, 2014 at 4:31 PM, Andres Freund wrote: > On 2014-09-10 12:17:34 +0530, Amit Kapila wrote: > > +++ b/src/backend/postmaster/bgreclaimer.c > > A fair number of comments in that file refer to bgwriter... Will fix. > > @@ -0,0 +1,302 @@ > > +/*-

Re: [HACKERS] proposal (9.5) : psql unicode border line styles

2014-09-11 Thread Pavel Stehule
2014-09-12 5:14 GMT+02:00 Stephen Frost : > Pavel, > > * Pavel Stehule (pavel.steh...@gmail.com) wrote: > > Any idea, tip how to it? > > Attached is what I had been thinking. > > Thoughts? > yes, it is better. I didn't understand before. I though about it because it allows to design multiline en

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-09-11 Thread Michael Paquier
On Mon, Feb 3, 2014 at 3:42 PM, Amit Kapila wrote: > On Wed, Jan 15, 2014 at 2:43 AM, Simon Riggs wrote: >> On 8 January 2014 08:33, Simon Riggs wrote: >> >> Patch attached, implemented to reduce writes by SELECTs only. This patch is registered in this CF. It does not apply anymore and needs a

Re: [HACKERS] Support for N synchronous standby servers

2014-09-11 Thread Michael Paquier
On Fri, Sep 12, 2014 at 12:48 AM, Robert Haas wrote: > On Wed, Sep 10, 2014 at 11:40 PM, Michael Paquier > wrote: >> Currently two nodes can only have the same priority if they have the >> same application_name, so we could for example add a new connstring >> parameter called, let's say applicati

Re: [HACKERS] vacuumdb --all --analyze-in-stages - wrong order?

2014-09-11 Thread Pavel Stehule
2014-09-12 3:44 GMT+02:00 Peter Eisentraut : > On 9/4/14 4:23 PM, Pavel Stehule wrote: > > It is little bit hard to read. > > > maybe better be more verbose - and it can be in alone function, because > > it is "analyze only" > > > > if (stage == -1) > > { > > for (i = 0; i < 3; i++) > > { > >

Re: [HACKERS] proposal (9.5) : psql unicode border line styles

2014-09-11 Thread Stephen Frost
Pavel, * Pavel Stehule (pavel.steh...@gmail.com) wrote: > Any idea, tip how to it? Attached is what I had been thinking. Thoughts? Thanks! Stephen diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml index aa71674..1d59dce 100644 --- a/doc/src/sg

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-09-11 Thread Alvaro Herrera
Thomas Munro wrote: > > Doesn't this heap_lock_tuple() need to check for a WouldBlock result? > > Not sure that this is the same case that you were trying to test in > > heap_lock_updated_tuple; I think this requires an update chain (so that > > EPQFetch is invoked) and some tuple later in the cha

Re: [HACKERS] pg_dump refactor patch to remove global variables

2014-09-11 Thread Alvaro Herrera
Joachim Wieland wrote: > On Sat, Aug 30, 2014 at 11:12 PM, Peter Eisentraut wrote: > > - Why is quote_all_identifiers left behind as a global variable? > > As I said, it's really used everywhere. I'm not opposed to passing it > around (which would be straightforward) but it would really blow up

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-11 Thread Arthur Silva
On Thu, Sep 11, 2014 at 10:01 PM, Josh Berkus wrote: > So, I finally got time to test Tom's latest patch on this. > > TLDR: we want to go with Tom's latest patch and release beta3. > > Figures: > > So I tested HEAD against the latest lengths patch. Per Arthur Silva, I > checked uncompressed time

Re: [HACKERS] vacuumdb --all --analyze-in-stages - wrong order?

2014-09-11 Thread Peter Eisentraut
On 9/4/14 4:23 PM, Pavel Stehule wrote: > It is little bit hard to read. > maybe better be more verbose - and it can be in alone function, because > it is "analyze only" > > if (stage == -1) > { > for (i = 0; i < 3; i++) > { > puts(gettext(stage_messages[i])); > executeCommand

Re: [HACKERS] Commitfest status

2014-09-11 Thread Tomonari Katsumata
Hi, I've update my entry. [rounding up time value less than its unit] https://commitfest.postgresql.org/action/patch_view?id=1507 regards, - Tomonari Katsumata (2014/09/12 7:03), Tomas Vondra wrote: > On 11.9.2014 21:14, Petr Jelinek wrote: >> On 11/09/14 18:59, Tomas Vondra wro

[HACKERS] Re: proposal: ignore null fields in not relation type composite type based constructors

2014-09-11 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote: > * Stephen Frost (sfr...@snowman.net) wrote: > > Ok. I'll handle updating both of these to remove the overloading > > and use default params instead, but I'll only add the 'ignore_null' > > option to row_to_json. > > Barring objections or any issues I

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-11 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: > TLDR: we want to go with Tom's latest patch and release beta3. Having not even read the rest- yes please. We really need to get beta3 out and figure out when we're going to actually release 9.4... Admittedly, the last month has been good and we've been f

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-11 Thread Josh Berkus
So, I finally got time to test Tom's latest patch on this. TLDR: we want to go with Tom's latest patch and release beta3. Figures: So I tested HEAD against the latest lengths patch. Per Arthur Silva, I checked uncompressed times for JSONB against compressed times. This changed the picture cons

Re: [HACKERS] Suspicious check (src/backend/access/gin/gindatapage.c)

2014-09-11 Thread Michael Paquier
On Fri, Sep 12, 2014 at 7:35 AM, Gaetano Mendola wrote: > At line 650 I can read: > > if ((leaf->lsize - segsize) - (leaf->lsize - segsize) < BLCKSZ / 4) > break; > > I believe one of the two should be leaf->rsize Yes this condition is broken. Shouldn't it be that instead when appending

Re: [HACKERS] [v9.5] Custom Plan API

2014-09-11 Thread Kouhei Kaigai
> On Thu, Sep 11, 2014 at 11:24 AM, Kouhei Kaigai > wrote: > >> Don't the changes to src/backend/optimizer/plan/createplan.c belong > >> in patch #2? > >> > > The borderline between #1 and #2 is little bit bogus. So, I moved most > > of portion into #1, however, invocation of InitCustomScan (that

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-11 Thread Peter Geoghegan
On Tue, Sep 9, 2014 at 2:25 PM, Robert Haas wrote: >> I like that I don't have to care about every combination, and can >> treat abbreviation abortion as the special case with the extra step, >> in line with how I think of the optimization conceptually. Does that >> make sense? > > No. comparetup

Re: [HACKERS] Incorrect initialization of sentPtr in walsender.c

2014-09-11 Thread Michael Paquier
On Fri, Sep 12, 2014 at 9:08 AM, Michael Paquier wrote: > In walsender.c, sentPtr is initialized as follows: > static XLogRecPtr sentPtr = 0; > Isn't that incorrect and shouldn't we use InvalidXLogRecPtr instead? Actually by looking more around I found a couple of extra places where the same incon

[HACKERS] Incorrect initialization of sentPtr in walsender.c

2014-09-11 Thread Michael Paquier
Hi all, In walsender.c, sentPtr is initialized as follows: static XLogRecPtr sentPtr = 0; Isn't that incorrect and shouldn't we use InvalidXLogRecPtr instead? Patch is attached. Regards, -- Michael diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 844a5d

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-09-11 Thread Thomas Munro
On 10 September 2014 14:47, Alvaro Herrera wrote: > Thomas Munro wrote: > >> @@ -2022,7 +2030,7 @@ EvalPlanQualFetch(EState *estate, Relation relation, >> int lockmode, bool noWait, >>*/ >> test = heap_lock_tuple(relation, &tuple, >>

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-09-11 Thread Stephen Frost
Tom, all, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Yeah, I've started looking at this patch and that seemed like not > necessarily such a wise choice. I think it'd be better if the generated > plan tree had a different structure in this case. The existing approach > is an impressive hack but it'

Re: [HACKERS] pg_upgrade and epoch

2014-09-11 Thread Bruce Momjian
On Thu, Sep 11, 2014 at 04:58:12PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, Sep 10, 2014 at 02:24:17AM +0100, Greg Stark wrote: > >> I think the reason nobody's responding is because nobody has anything > >> significant to add. It's a behavior change from not-working to > >> work

[HACKERS] Suspicious check (src/backend/access/gin/gindatapage.c)

2014-09-11 Thread Gaetano Mendola
At line 650 I can read: if ((leaf->lsize - segsize) - (leaf->lsize - segsize) < BLCKSZ / 4) break; I believe one of the two should be leaf->rsize -- cpp-today.blogspot.com

[HACKERS] Re: proposal: ignore null fields in not relation type composite type based constructors

2014-09-11 Thread Stephen Frost
All, * Stephen Frost (sfr...@snowman.net) wrote: > Ok. I'll handle updating both of these to remove the overloading > and use default params instead, but I'll only add the 'ignore_null' > option to row_to_json. Barring objections or any issues I find as I go back through it, this is what I'm pla

Re: [HACKERS] Commitfest status

2014-09-11 Thread Tomas Vondra
On 11.9.2014 21:14, Petr Jelinek wrote: > On 11/09/14 18:59, Tomas Vondra wrote: >> On 10.9.2014 22:39, Heikki Linnakangas wrote: >>> The bad news is that the rest don't seem to moving graduating from the >>> Needs Review state. >> >> ISTM that many patches >> >> (b) in 'waiting on author' are not

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tomas Vondra
On 11.9.2014 16:33, Tomas Vondra wrote: > On 11 Září 2014, 15:31, Robert Haas wrote: >> On Wed, Sep 10, 2014 at 5:09 PM, Tomas Vondra wrote: >>> OK. So here's v13 of the patch, reflecting this change. >> >> [...] It does three things: >> >> (1) It changes NTUP_PER_BUCKET to 1. Although this incre

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-11 Thread Peter Geoghegan
On Thu, Sep 11, 2014 at 1:50 PM, Robert Haas wrote: > I think I said pretty clearly that it was. I agree that you did, but it wasn't clear exactly what factors you were asking me to simulate. It still isn't. Do you want me to compare the same string a million times in a loop, both with a strcoll(

Re: [HACKERS] pg_upgrade and epoch

2014-09-11 Thread Andres Freund
On 2014-09-11 16:58:12 -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, Sep 10, 2014 at 02:24:17AM +0100, Greg Stark wrote: > >> I think the reason nobody's responding is because nobody has anything > >> significant to add. It's a behavior change from not-working to > >> working. Why wou

Re: [HACKERS] pg_upgrade and epoch

2014-09-11 Thread Tom Lane
Bruce Momjian writes: > On Wed, Sep 10, 2014 at 02:24:17AM +0100, Greg Stark wrote: >> I think the reason nobody's responding is because nobody has anything >> significant to add. It's a behavior change from not-working to >> working. Why wouldn't it be backpatched? > OK, Greg seems to be passion

Re: [HACKERS] pg_upgrade and epoch

2014-09-11 Thread Bruce Momjian
On Wed, Sep 10, 2014 at 02:24:17AM +0100, Greg Stark wrote: > On Tue, Sep 9, 2014 at 4:05 PM, Bruce Momjian wrote: > >> > Yes, I did think about that, but it seems like a behavior change. > >> > However, it is tempting to avoid future bug reports about this. > >> > >> When this came up in March, T

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 4:13 PM, Peter Geoghegan wrote: > On Wed, Sep 10, 2014 at 11:36 AM, Robert Haas wrote: >> No, not really. All you have to do is right a little test program to >> gather the information. > > I don't think a little test program is useful - IMV it's too much of a > simplific

Re: [HACKERS] Stating the significance of Lehman & Yao in the nbtree README

2014-09-11 Thread Peter Geoghegan
On Tue, Sep 9, 2014 at 12:01 AM, Heikki Linnakangas wrote: >> + Lehman and Yao don't require read locks, but assume that in-memory >> + copies of tree pages are unshared. > This is the existing paragraph, just moved to different place in the README. That's right - it seemed to make just as much

Re: [HACKERS] RLS Design

2014-09-11 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Sep 11, 2014 at 3:08 PM, Stephen Frost wrote: > > The one thing I'm wondering about with this design is- what happens when > > a policy is initially added? Currently, we automatically turn on RLS > > for the table when that happens. I'm not

Re: [HACKERS] RLS Design

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 3:08 PM, Stephen Frost wrote: >> 2. Row level security policies can exist for a table with DISABLE ROW >> LEVEL SECURITY in effect, but they don't do anything until RLS is >> enabled. A possible advantage of this approach is that you could >> *temporarily* shut off RLS for

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-11 Thread Peter Geoghegan
On Wed, Sep 10, 2014 at 11:36 AM, Robert Haas wrote: > No, not really. All you have to do is right a little test program to > gather the information. I don't think a little test program is useful - IMV it's too much of a simplification to suppose that a strcoll() has a fixed cost, and a memcmp()

Re: [HACKERS] [v9.5] Custom Plan API

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 11:24 AM, Kouhei Kaigai wrote: >> Don't the changes to src/backend/optimizer/plan/createplan.c belong in >> patch #2? >> > The borderline between #1 and #2 is little bit bogus. So, I moved most of > portion into #1, however, invocation of InitCustomScan (that is a callback

Re: [HACKERS] Aussie timezone database changes incoming

2014-09-11 Thread Gavin Flower
On 12/09/14 01:57, Robert Haas wrote: On Thu, Sep 11, 2014 at 12:20 AM, Craig Ringer wrote: I shouldn't be surprised that Australia gets to change. While the cynic in me thinks this is the usual USA-is-the-center-of-the-universe-ism, in reality it makes sense given relative population and likel

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-11 Thread Andres Freund
On 2014-09-11 21:27:15 +0200, Petr Jelinek wrote: > On 11/09/14 20:37, Robert Haas wrote: > >OK. I still think we should go back and PGDLLIMPORT-ize all the GUC > >variables. > > +1 Same here. I think Tom was the only one against it, but pretty much everyone else was for it. We should fix all t

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-11 Thread Petr Jelinek
On 11/09/14 20:37, Robert Haas wrote: 1. This patch generates warning on windows 1>pg_background.obj : error LNK2001: unresolved external symbol StatementTimeout You need to add PGDLLIMPORT for StatementTimeout OK. I still think we should go back and PGDLLIMPORT-ize all the GUC variables. +

Re: [HACKERS] Commitfest status

2014-09-11 Thread Petr Jelinek
On 11/09/14 18:59, Tomas Vondra wrote: On 10.9.2014 22:39, Heikki Linnakangas wrote: The bad news is that the rest don't seem to moving graduating from the Needs Review state. ISTM that many patches (b) in 'waiting on author' are not really waiting, because the author already responded /

Re: [HACKERS] RLS Design

2014-09-11 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Sat, Sep 6, 2014 at 2:54 AM, Brightwell, Adam > wrote: > > * Add ALTER TABLE { ENABLE | DISABLE } ROW LEVEL SECURITY - set flag > > on table to allow for a default-deny capability. If RLS is enabled on a > > table and has no policies, th

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-11 Thread Fabien COELHO
However, that would not diminish nor change much the amount and kind of code necessary to add an operator or a function That's not really true. You can't really add abs(x) or hash(x) right now because the current code only supports this syntax: \set varname operand1 [ operator operand2 ] Th

Re: [HACKERS] about half processes are blocked by btree, btree is bottleneck?

2014-09-11 Thread Peter Geoghegan
On Wed, Sep 10, 2014 at 8:08 PM, Xiaoyulei wrote: > Sum:66 > #0 0x7f8273a77627 in semop () from /lib64/libc.so.6 > #1 0x0060cda7 in PGSemaphoreLock () > #2 0x006511a9 in LWLockAcquire () > #3 0x004987f7 in _bt_relandgetbuf () > #4 0x0049c116 in _bt_search (

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread k...@rice.edu
On Thu, Sep 11, 2014 at 02:54:36PM -0300, Arthur Silva wrote: > Indeed I don't know any other architectures that this would be at an > option. So if this ever moves forward it must be turned on at compile time > for x86-64 only. I wonder how the Mysql handle their rows even on those > architectures

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 7:34 AM, Amit Kapila wrote: > Don't we need some way to prohibit changing GUC by launching process, > once it has shared the existing GUC? Nope. I mean, eventually, for true parallelism ... we absolutely will need that. But pg_background itself doesn't need that; it's pe

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread k...@rice.edu
On Thu, Sep 11, 2014 at 07:17:42PM +0200, Andres Freund wrote: > On 2014-09-11 13:04:43 -0400, Robert Haas wrote: > > On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund > > wrote: > > > On 2014-09-11 12:55:21 -0400, Robert Haas wrote: > > >> I advise supporting pglz only for the initial patch, and a

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread k...@rice.edu
On Thu, Sep 11, 2014 at 06:58:06PM +0200, Andres Freund wrote: > On 2014-09-11 12:55:21 -0400, Robert Haas wrote: > > I advise supporting pglz only for the initial patch, and adding > > support for the others later if it seems worthwhile. The approach > > seems to work well enough with pglz that i

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 1:17 PM, Andres Freund wrote: > On 2014-09-11 13:04:43 -0400, Robert Haas wrote: >> On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund >> wrote: >> > On 2014-09-11 12:55:21 -0400, Robert Haas wrote: >> >> I advise supporting pglz only for the initial patch, and adding >> >>

[HACKERS] Re: proposal: ignore null fields in not relation type composite type based constructors

2014-09-11 Thread Stephen Frost
Andrew, * Andrew Dunstan (and...@dunslane.net) wrote: > On 09/11/2014 08:29 AM, Stephen Frost wrote: > >* Pavel Stehule (pavel.steh...@gmail.com) wrote: > >>Can I help with something, it is there some open question? > >I had been hoping for a more definitive answer regarding this option for > >arr

Re: [HACKERS] proposal (9.5) : psql unicode border line styles

2014-09-11 Thread Stephen Frost
* Pavel Stehule (pavel.steh...@gmail.com) wrote: > 2014-09-11 16:42 GMT+02:00 Stephen Frost : > > I don't particularly like this (having these fields set in > > refresh_utf8format to hard-coded strings in the function), why not have > > those handled the same as the rest, where the strings themselv

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Arthur Silva
On Thu, Sep 11, 2014 at 12:39 PM, Tom Lane wrote: > Merlin Moncure writes: > > Be advised of the difficulties you are going to face here. Assuming > > for a second there is no reason not to go unaligned on Intel and there > > are material benefits to justify the effort, that doesn't necessarily

Re: [HACKERS] implement subject alternative names support for SSL connections

2014-09-11 Thread Alexey Klyukin
On Mon, Sep 8, 2014 at 8:04 PM, Heikki Linnakangas wrote: > On 09/05/2014 07:30 PM, Alexey Klyukin wrote: >> The error does not state whether the names comes from the CN or from >> the SAN section. > > > I'd reword that slightly, to: > > psql: server certificate for "example.com" (and 2 other nam

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Andres Freund
On 2014-09-11 13:04:43 -0400, Robert Haas wrote: > On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund > wrote: > > On 2014-09-11 12:55:21 -0400, Robert Haas wrote: > >> I advise supporting pglz only for the initial patch, and adding > >> support for the others later if it seems worthwhile. The appr

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Arthur Silva
On Thu, Sep 11, 2014 at 11:27 AM, Andres Freund wrote: > On 2014-09-11 10:32:24 -0300, Arthur Silva wrote: > > Unaligned memory access received a lot attention in Intel post-Nehalen > era. > > So it may very well pay off on Intel servers. You might find this blog > post > > and it's comments/exte

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 2:47 AM, Fabien COELHO wrote: > Ok. I do agree that an expression syntax would be great! > > However, that would not diminish nor change much the amount and kind of code > necessary to add an operator or a function That's not really true. You can't really add abs(x) or ha

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund wrote: > On 2014-09-11 12:55:21 -0400, Robert Haas wrote: >> I advise supporting pglz only for the initial patch, and adding >> support for the others later if it seems worthwhile. The approach >> seems to work well enough with pglz that it's worth

Re: [HACKERS] Commitfest status

2014-09-11 Thread Tomas Vondra
On 10.9.2014 22:39, Heikki Linnakangas wrote: > The bad news is that the rest don't seem to moving graduating from the > Needs Review state. ISTM that many patches (a) in 'needs review' actually have a review, or are being thoroughly discussed (b) in 'waiting on author' are not really waitin

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Bruce Momjian
On Thu, Sep 11, 2014 at 12:55:21PM -0400, Robert Haas wrote: > I advise supporting pglz only for the initial patch, and adding > support for the others later if it seems worthwhile. The approach > seems to work well enough with pglz that it's worth doing even if we > never add the other algorithms

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Andres Freund
On 2014-09-11 12:55:21 -0400, Robert Haas wrote: > I advise supporting pglz only for the initial patch, and adding > support for the others later if it seems worthwhile. The approach > seems to work well enough with pglz that it's worth doing even if we > never add the other algorithms. That appr

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 1:46 AM, Rahila Syed wrote: >>I will repeat the above tests with high load on CPU and using the benchmark > given by Fujii-san and post the results. > > Average % of CPU usage at user level for each of the compression algorithm > are as follows. > > CompressionMulti

Re: [HACKERS] RLS Design

2014-09-11 Thread Robert Haas
On Sat, Sep 6, 2014 at 2:54 AM, Brightwell, Adam wrote: > * Add ALTER TABLE { ENABLE | DISABLE } ROW LEVEL SECURITY - set flag > on table to allow for a default-deny capability. If RLS is enabled on a > table and has no policies, then a default-deny policy is automatically > applied. If RLS is

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tomas Vondra
On 11 Září 2014, 17:28, Tom Lane wrote: > "Tomas Vondra" writes: >> On 11 Z?? 2014, 16:11, Tom Lane wrote: >>> Ah. Well, that would mean that we need a heuristic for deciding when >>> to >>> increase the number of buckets versus the number of batches ... seems >>> like a difficult decision. >

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-09-11 Thread Tom Lane
Stephen Frost writes: > I have to admit that, while I applaud the effort made to have this > change only be to postgres_fdw, I'm not sure that having the > update/delete happening during the Scan phase and then essentially > no-op'ing the ExecForeignUpdate/ExecForeignDelete is entirely in-line > w

Re: [HACKERS] proposal (9.5) : psql unicode border line styles

2014-09-11 Thread Pavel Stehule
Hi 2014-09-11 16:42 GMT+02:00 Stephen Frost : > Pavel, > > * Pavel Stehule (pavel.steh...@gmail.com) wrote: > > I removed dynamic allocation and reduced patch size. > > This is certainly better, imv, though there are a couple of minor > issues (extra semi-colons, extraneous whitespace, get_line_

Re: [HACKERS] Support for N synchronous standby servers

2014-09-11 Thread Robert Haas
On Wed, Sep 10, 2014 at 11:40 PM, Michael Paquier wrote: > Currently two nodes can only have the same priority if they have the > same application_name, so we could for example add a new connstring > parameter called, let's say application_group, to define groups of > nodes that will have the same

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-11 Thread Tom Lane
Robert Haas writes: > On Thu, Sep 11, 2014 at 7:14 AM, David Rowley wrote: >> 5. I've added a flag to pg_class called relhasfkey. Currently this gets set >> to true when a foreign key is added, though I've added nothing to set it >> back to false again. I notice that relhasindex gets set back to

[HACKERS] Re: proposal: ignore null fields in not relation type composite type based constructors

2014-09-11 Thread Andrew Dunstan
On 09/11/2014 08:29 AM, Stephen Frost wrote: * Pavel Stehule (pavel.steh...@gmail.com) wrote: Can I help with something, it is there some open question? I had been hoping for a more definitive answer regarding this option for array_to_json, or even a comment about the change to row_to_json. An

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Andres Freund
On 2014-09-11 11:39:12 -0400, Tom Lane wrote: > Even on Intel, I'd wonder what unaligned accesses do to atomicity > guarantees and suchlike. They pretty much kill atomicity guarantees. Atomicity is guaranteed while you're inside a cacheline, but not once you span them. > This is not a big deal fo

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-11 Thread Petr Jelinek
On 10/09/14 22:53, Robert Haas wrote: Here's what the other approach looks like. I can't really see doing this way and then only providing hooks for those two functions, so this is with hooks for all the send-side stuff. Original version: 9 files changed, 295 insertions(+), 3 deletions(-) This

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Tom Lane
Merlin Moncure writes: > Be advised of the difficulties you are going to face here. Assuming > for a second there is no reason not to go unaligned on Intel and there > are material benefits to justify the effort, that doesn't necessarily > hold for other platforms like arm/power. Note that on ma

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 7:14 AM, David Rowley wrote: > Here's a quick demo, of the patch at work: > > test=# create table c (id int primary key); > CREATE TABLE > test=# create table b (id int primary key, c_id int not null references > c(id)); > CREATE TABLE > test=# create table a (id int primar

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-09-11 Thread Stephen Frost
* Albe Laurenz (laurenz.a...@wien.gv.at) wrote: > Etsuro Fujita wrote: > > I agree with you on that point. So, I've updated the patch to have the > > explicit flag, as you proposed. Attached is the updated version of the > > patch. In this version, I've also revised code and its comments a bit.

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tom Lane
"Tomas Vondra" writes: > On 11 Září 2014, 16:11, Tom Lane wrote: >> Ah. Well, that would mean that we need a heuristic for deciding when to >> increase the number of buckets versus the number of batches ... seems >> like a difficult decision. > That's true, but that's not the aim of this patc

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 10:11 AM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Sep 11, 2014 at 9:59 AM, Tom Lane wrote: >>> Robert Haas writes: (3) It allows the number of batches to increase on the fly while the hash join is in process. > >>> Pardon me for not having read the pat

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Merlin Moncure
On Thu, Sep 11, 2014 at 8:32 AM, Arthur Silva wrote: > > On Wed, Sep 10, 2014 at 12:43 PM, Robert Haas wrote: >> >> On Tue, Sep 9, 2014 at 10:08 AM, Arthur Silva wrote: >> > I'm continuously studying Postgres codebase. Hopefully I'll be able to >> > make >> > some contributions in the future. >>

Re: [HACKERS] Add shutdown_at_recovery_target option to recovery.conf

2014-09-11 Thread Petr Jelinek
On 10/09/14 13:13, Fujii Masao wrote: On Wed, Sep 10, 2014 at 1:45 AM, Petr Jelinek wrote: Hi, I recently wanted several times to have slave server prepared at certain point in time to reduce the time it takes for it to replay remaining WALs (say I have pg_basebackup -x on busy db for example)

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tomas Vondra
On 11 Září 2014, 16:11, Tom Lane wrote: > Robert Haas writes: >> On Thu, Sep 11, 2014 at 9:59 AM, Tom Lane wrote: >>> Robert Haas writes: (3) It allows the number of batches to increase on the fly while the hash join is in process. > >>> Pardon me for not having read the patch yet, but

Re: [HACKERS] proposal (9.5) : psql unicode border line styles

2014-09-11 Thread Stephen Frost
Pavel, * Pavel Stehule (pavel.steh...@gmail.com) wrote: > I removed dynamic allocation and reduced patch size. This is certainly better, imv, though there are a couple of minor issues (extra semi-colons, extraneous whitespace, get_line_style was still changed to non-const, even though it doesn't

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tomas Vondra
On 11 Září 2014, 15:31, Robert Haas wrote: > On Wed, Sep 10, 2014 at 5:09 PM, Tomas Vondra wrote: >> OK. So here's v13 of the patch, reflecting this change. > > With the exception of ExecChooseHashTableSize() and a lot of stylistic > issues along the lines of what I've already complained about, th

[HACKERS] gist vacuum seaq access

2014-09-11 Thread Костя Кузнецов
After discussion of gist seaq access in vaccum there are 2 issue: Heikki says :Vacuum needs to memorize the current NSN when it begins1) how i may getting corect NSN. Also i must setting F_DELETED flag on empty page and to clean parent from link on deleted_pages2) how i may getting parent of page f

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Andres Freund
On 2014-09-11 10:32:24 -0300, Arthur Silva wrote: > Unaligned memory access received a lot attention in Intel post-Nehalen era. > So it may very well pay off on Intel servers. You might find this blog post > and it's comments/external-links interesting > http://lemire.me/blog/archives/2012/05/31/da

Re: [HACKERS] pgbench throttling latency limit

2014-09-11 Thread Fabien COELHO
How should skipped transactions should be taken into account in the log file output, with and without aggregation? I assume we'll want to have some trace of skipped transactions in the logs. The problem with this point is that how to report something "not done" is unclear, especially as the

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 10:03 AM, Andres Freund wrote: > On 2014-09-11 09:48:10 -0400, Robert Haas wrote: >> On Thu, Sep 11, 2014 at 9:22 AM, Andres Freund >> wrote: >> > I wonder if we should recheck the number of freelist items before >> > sleeping. As the latch currently is reset before sleep

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tom Lane
Robert Haas writes: > On Thu, Sep 11, 2014 at 9:59 AM, Tom Lane wrote: >> Robert Haas writes: >>> (3) It allows the number of batches to increase on the fly while the >>> hash join is in process. >> Pardon me for not having read the patch yet, but what part of (3) >> wasn't there already? > EI

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 9:59 AM, Tom Lane wrote: > Robert Haas writes: >> With the exception of ExecChooseHashTableSize() and a lot of stylistic >> issues along the lines of what I've already complained about, this >> patch seems pretty good to me. It does three things: >> ... >> (3) It allows t

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Andres Freund
On 2014-09-11 09:48:10 -0400, Robert Haas wrote: > On Thu, Sep 11, 2014 at 9:22 AM, Andres Freund wrote: > > I wonder if we should recheck the number of freelist items before > > sleeping. As the latch currently is reset before sleeping (IIRC) we > > might miss being woken up soon. It very well mi

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tom Lane
Robert Haas writes: > With the exception of ExecChooseHashTableSize() and a lot of stylistic > issues along the lines of what I've already complained about, this > patch seems pretty good to me. It does three things: > ... > (3) It allows the number of batches to increase on the fly while the > h

Re: [HACKERS] Aussie timezone database changes incoming

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 12:20 AM, Craig Ringer wrote: > I shouldn't be surprised that Australia gets to change. While the cynic > in me thinks this is the usual USA-is-the-center-of-the-universe-ism, in > reality it makes sense given relative population and likely impact. Just because it makes se

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-11 Thread Mitsumasa KONDO
2014-09-11 15:47 GMT+09:00 Fabien COELHO : > > Hello Robert, > > I am not objecting to the functionality; I'm objecting to bolting on >> ad-hoc operators one at a time. I think an expression syntax would >> let us do this in a much more scalable way. If I had time, I'd go do >> that, but I don'

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 9:32 AM, Arthur Silva wrote: > I thought all memory alignment was (or at least the bulk of it) handled > using some codebase wide macros/settings, otherwise how could different > parts of the code inter-op? Poking this area might suffice for some initial > testing to check

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 9:22 AM, Andres Freund wrote: >> It's exactly the same as what bgwriter.c does. > > So what? There's no code in common, so I see no reason to have one > signal handler using underscores and the next one camelcase names. /me shrugs. It's not always possible to have things

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Amit Kapila
On Thu, Sep 11, 2014 at 6:59 PM, Andres Freund wrote: > > > > We really need a more centralized way to handle error cleanup in > > > auxiliary processes. The current state of affairs is really pretty > > > helter-skelter. But for this patch, I think we should aim to mimic > > > the existing styl

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Mitsumasa KONDO
2014-09-11 22:01 GMT+09:00 k...@rice.edu : > On Thu, Sep 11, 2014 at 09:37:07AM -0300, Arthur Silva wrote: > > I agree that there's no reason to fix an algorithm to it, unless maybe > it's > > pglz. > Yes, it seems difficult to judge only the algorithm performance. We have to start to consider so

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Arthur Silva
On Wed, Sep 10, 2014 at 12:43 PM, Robert Haas wrote: > On Tue, Sep 9, 2014 at 10:08 AM, Arthur Silva wrote: > > I'm continuously studying Postgres codebase. Hopefully I'll be able to > make > > some contributions in the future. > > > > For now I'm intrigued about the extensive use of memory alig

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Robert Haas
On Wed, Sep 10, 2014 at 5:09 PM, Tomas Vondra wrote: > OK. So here's v13 of the patch, reflecting this change. With the exception of ExecChooseHashTableSize() and a lot of stylistic issues along the lines of what I've already complained about, this patch seems pretty good to me. It does three th

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Andres Freund
> > We really need a more centralized way to handle error cleanup in > > auxiliary processes. The current state of affairs is really pretty > > helter-skelter. But for this patch, I think we should aim to mimic > > the existing style, as ugly as it is. I'm not sure whether Amit's got > > the log

Re: [HACKERS] pgbench throttling latency limit

2014-09-11 Thread Fabien COELHO
Hello Heikki Now that I've finished the detour and committed and backpatched the changes to the way latency is calculated, we can get back to this patch. It needs to be rebased. Here is the rebase, which seems ok. See also the small issues raised aboud getPoissonRand in another email. -- F

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Andres Freund
On 2014-09-11 09:02:34 -0400, Robert Haas wrote: > Thanks for reviewing, Andres. > > On Thu, Sep 11, 2014 at 7:01 AM, Andres Freund wrote: > >> +static void bgreclaim_quickdie(SIGNAL_ARGS); > >> +static void BgreclaimSigHupHandler(SIGNAL_ARGS); > >> +static void ReqShutdownHandler(SIGNAL_ARGS); >

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Amit Kapila
On Thu, Sep 11, 2014 at 6:32 PM, Robert Haas wrote: > > Thanks for reviewing, Andres. > > On Thu, Sep 11, 2014 at 7:01 AM, Andres Freund wrote: > >> +static void bgreclaim_quickdie(SIGNAL_ARGS); > >> +static void BgreclaimSigHupHandler(SIGNAL_ARGS); > >> +static void ReqShutdownHandler(SIGNAL_ARG

  1   2   >