Re: [HACKERS] moving from contrib to bin

2015-01-14 Thread Michael Paquier
On Tue, Dec 23, 2014 at 3:24 PM, Michael Paquier wrote: > On Mon, Dec 22, 2014 at 11:30 PM, Alvaro Herrera > wrote: >> Michael Paquier wrote: >> >>> And here is an updated patch following those lines. Similarly to the >>> things in contrib/, a set of variables are used to define for each >>> modu

Re: [HACKERS] DROP PRIVILEGES OWNED BY

2015-01-14 Thread Michael Paquier
On Thu, Dec 18, 2014 at 1:43 AM, Marko Tiikkaja wrote: > I don't have a problem with that. It would probably work, too, since FROM > is already fully reserved. Marking patch as returned with feedback as there has been no input from Marko in the last couple of weeks. -- Michael -- Sent via pgs

Re: [HACKERS] Support UPDATE table SET(*)=...

2015-01-14 Thread Michael Paquier
On Mon, Dec 15, 2014 at 7:50 PM, Atri Sharma wrote: > I have moved patch to current CF and marked it as Waiting on Author since I > plan to resubmit after addressing the concerns. Nothing happened in the last month, so marking as returned with feedback. -- Michael -- Sent via pgsql-hackers mai

Re: [HACKERS] Async execution of postgres_fdw.

2015-01-14 Thread Michael Paquier
On Tue, Jan 13, 2015 at 8:46 PM, Kyotaro HORIGUCHI wrote: > I'll look into the case after this, but I'd like to send a > revised patch at this point. Hm. Seems like this patch is not completely baked yet. Horiguchi-san, as you are obviously still working on it, would you agree to move it to the ne

Re: [HACKERS] logical column ordering

2015-01-14 Thread Michael Paquier
On Sun, Jan 4, 2015 at 10:37 AM, Alvaro Herrera wrote: > So I'm reworking my patch with that in mind. Switching to returned with feedback. Alvaro, feel free to add an entry to the next CF if you are planning to work on it again. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@

Re: [HACKERS] Join push-down support for foreign tables

2015-01-14 Thread Michael Paquier
On Fri, Dec 26, 2014 at 1:48 PM, Shigeru Hanada wrote: > Hmm, I agree to support N-way join is very useful. Postgres-XC's SQL > generator seems to give us a hint for such case, I'll check it out > again. Switching to returned with feedback, as this patch is waiting for feedback for a couple of we

Re: [HACKERS] install libpq.dll in bin directory on Windows / Cygwin

2015-01-14 Thread Michael Paquier
On Wed, Dec 24, 2014 at 3:08 PM, Michael Paquier wrote: > Attached are two patches, one for MinGW/cygwin, a slightly modified > version from Peter and the second implementing the same thing but for > the MSVC scripts. The method for MSVC is similar to what is done in > Peter's patch: roughly it ch

Re: [HACKERS] inherit support for foreign tables

2015-01-14 Thread Etsuro Fujita
On 2014/12/23 0:36, Tom Lane wrote: > Yeah, we need to do something about the PlanRowMark data structure. > Aside from the pre-existing issue in postgres_fdw, we need to fix it > to support inheritance trees in which more than one rowmark method > is being used. That rte.hasForeignChildren thing i

Re: [HACKERS] patch : Allow toast tables to be moved to a different tablespace

2015-01-14 Thread Michael Paquier
On Tue, Dec 30, 2014 at 11:48 AM, Andreas Karlsson wrote: > Here is my review of the feature. Marked as returned with feedback. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hacker

Re: [HACKERS] compress method for spgist - 2

2015-01-14 Thread Michael Paquier
Marking this patch as returned with feedback because it is waiting for input from the author for now a couple of weeks. Heikki, the refactoring patch has some value, are you planning to push it? -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] Compression of full-page-writes

2015-01-14 Thread Michael Paquier
Marking this patch as returned with feedback for this CF, moving it to the next one. I doubt that there will be much progress here for the next couple of days, so let's try at least to get something for this release cycle. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-14 Thread Michael Paquier
On Thu, Jan 8, 2015 at 2:03 PM, Michael Paquier wrote: > On Thu, Jan 8, 2015 at 6:31 AM, Alexander Korotkov > wrote: >> On Wed, Jan 7, 2015 at 4:11 PM, Michael Paquier >>> I am attaching an updated patch, with the default fillfactor value at >>> 75%, and with the page split code using the fillf

Re: [HACKERS] [PATCH] add ssl_protocols configuration option

2015-01-14 Thread Michael Paquier
On Mon, Dec 15, 2014 at 11:15 PM, Alex Shulgin wrote: > Michael Paquier writes: Perhaps ssloptions.[ch], unless you plan to add non-option-related code there later? >>> >>> I don't think anything else than common options-related code fits in >>> there, so ssloptions.c makes sense t

[HACKERS] EvalPlanQual behaves oddly for FDW queries involving system columns

2015-01-14 Thread Etsuro Fujita
Here is an example using postgres_fdw. [Terminal 1] postgres=# create table t (a int, b int); CREATE TABLE postgres=# insert into t values (1, 1); INSERT 0 1 postgres=# begin; BEGIN postgres=# update t set b = b * 2; UPDATE 1 [Terminal 2] postgres=# create foreign table ft (a int) server loopback

Re: [HACKERS] __attribute__ for non-gcc compilers

2015-01-14 Thread Oskari Saarenmaa
15.01.2015, 00:54, Andres Freund kirjoitti: > I think I'd for now simply not define pg_attribute_aligned() on > platforms where it's not supported, instead of defining it empty. If we > need a softer variant we can name it pg_attribute_aligned_if_possible or > something. Good point, all attributes

[HACKERS] Commit fest 2015-12 enters money time

2015-01-14 Thread Michael Paquier
Hi all, We are soon entering in the money time for this CF. The last month has been mainly a vacation period, the progress being fantomatic on many fronts, still there are a couple of patches that are marked as ready for committer: - Foreign table inheritance , whose first patch has been committed

Re: [HACKERS] PATCH: Reducing lock strength of trigger and foreign key DDL

2015-01-14 Thread Michael Paquier
I wrote: > I think that we could pass it to a committer. Marked as such. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Define and document a guaranteed ordering for RETURNING?

2015-01-14 Thread Craig Ringer
Apparently I'm semi-blind - the docs also note that: > If the INSERT command contains a RETURNING clause, the result will be similar to that of a SELECT statement containing the columns and values defined in the RETURNING list, computed over the row(s) inserted by the command. ... so perhaps it's

Re: [HACKERS] Overhauling our interrupt handling

2015-01-14 Thread Kyotaro HORIGUCHI
Hello, I'd synced up this at last. I think I should finilize my commitfest item for this issue, with .. "Rejected"? > This mail is a answer to > http://archives.postgresql.org/message-id/20150110022542.GF12509%40alap3.anarazel.de > but I decided that it's a good go move it into a new thread since

Re: [HACKERS] orangutan seizes up during isolation-check

2015-01-14 Thread Noah Misch
On Wed, Jan 14, 2015 at 04:48:53PM -0500, Peter Eisentraut wrote: > What I'm seeing now is that the unaccent regression tests when run under > make check-world abort with > > FATAL: postmaster became multithreaded during startup > HINT: Set the LC_ALL environment variable to a valid locale. con

Re: [HACKERS] Parallel Seq Scan

2015-01-14 Thread Amit Kapila
On Wed, Jan 14, 2015 at 9:30 AM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > > On Wed, Jan 14, 2015 at 9:12 AM, Amit Kapila wrote: >> >> Here we have to decide what should be the strategy and how much >> each worker should scan. As an example one of the the strategy >> could be if

[HACKERS] Define and document a guaranteed ordering for RETURNING?

2015-01-14 Thread Craig Ringer
Hi all It's recently occurred to me that many people rely on the RETURNING clause for multi-valued INSERTs, INSERT INTO ... SELECT, etc, returning rows in the same order they were supplied as input into the INSERT. I often see lock-step iteration over the inputs to an INSERT and the RETURNING resu

Re: [HACKERS] initdb -S and tablespaces

2015-01-14 Thread Abhijit Menon-Sen
At 2015-01-14 11:59:08 +0100, and...@2ndquadrant.com wrote: > > > + if (ControlFile->state != DB_SHUTDOWNED && > > + ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY) > > + perform_fsync(data_directory); > > + > > a) Please think of a slightly more descriptive name than perfor

Re: [HACKERS] pg_rewind in contrib

2015-01-14 Thread Michael Paquier
On Wed, Jan 14, 2015 at 6:43 PM, Heikki Linnakangas wrote: > Here is a new version. There are now a fair amount of code changes compared > to the version on github, like the logging and progress information, and a > lot of comment changes. I also improved the documentation. Perhaps this could a d

Re: [HACKERS] Merging postgresql.conf and postgresql.auto.conf

2015-01-14 Thread Amit Kapila
On Wed, Jan 14, 2015 at 9:01 PM, Sawada Masahiko wrote: > > Hi all, > > The postgresql.auto.conf is loaded after loading of postgresql.conf > whenever configuration file is loaded or reloaded. > This means that parameter in postgresql.auto.conf is quite high > priority, so the parameter in postgre

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Peter Geoghegan
On Wed, Jan 14, 2015 at 5:23 PM, Peter Geoghegan wrote: > My immediate observation here is that blocks 2 and 9 have identical > metadata (from their page opaque area), but partially non-matching > data items (however, the number of items on each block is consistent > and correct according to that

[HACKERS] Enforce creation of destination folders for source files in pg_regress (Was: pg_regress writes into source tree)

2015-01-14 Thread Michael Paquier
Hi all, pg_regress will fail with test suites using only source files if the destination folders do not exist in the code tree. This is annoying because this forces to maintain empty folders sql/ and expected/ with a .gitignore ignoring everything. The issue has been discussed here (http://www.pos

Re: [HACKERS] WITH CHECK and Column-Level Privileges

2015-01-14 Thread Stephen Frost
Noah, * Noah Misch (n...@leadboat.com) wrote: > On Mon, Jan 12, 2015 at 05:16:40PM -0500, Stephen Frost wrote: > > Alright, here's an updated patch which doesn't return any detail if no > > values are visible or if only a partial key is visible. > > I browsed this patch. There's been no mention

Re: [HACKERS] Improving RLS qual pushdown

2015-01-14 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Robert Haas wrote: > > On Wed, Jan 14, 2015 at 9:22 AM, Dean Rasheed > > wrote: > > > On 14 January 2015 at 13:29, Robert Haas wrote: > > >> One thing they could still leak is the number of times they got > > >> called, and thus possibly the n

Re: [HACKERS] Improving RLS qual pushdown

2015-01-14 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Jan 14, 2015 at 9:22 AM, Dean Rasheed > wrote: > > On 14 January 2015 at 13:29, Robert Haas wrote: > >> One thing they could still leak is the number of times they got > >> called, and thus possibly the number of unseen rows. Now if the > >> expressions get constant

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Peter Geoghegan
On Wed, Jan 14, 2015 at 5:23 PM, Peter Geoghegan wrote: > My immediate observation here is that blocks 2 and 9 have identical > metadata (from their page opaque area), but partially non-matching > data items (however, the number of items on each block is consistent > and correct according to that

Re: [HACKERS] Safe memory allocation functions

2015-01-14 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Jan 13, 2015 at 10:10 AM, Tom Lane wrote: > > However, there is a larger practical problem with this whole concept, > > which is that experience should teach us to be very wary of the assumption > > that asking for memory the system can't give us will just lead to nice

Re: [HACKERS] Out-of-bounds write and incorrect detection of trigger file in pg_standby

2015-01-14 Thread Michael Paquier
On Thu, Jan 15, 2015 at 7:13 AM, Robert Haas wrote: > Instead of doing this: > > if (len < sizeof(buf)) > buf[len] = '\0'; > > ...I would suggest making the size of the buffer one greater than the > size of the read(), and then always nul-terminating the buffer. It > seems to me that

Partitioning: issues/ideas (Was: Re: [HACKERS] On partitioning)

2015-01-14 Thread Amit Langote
On 06-01-2015 PM 03:40, Amit Langote wrote: > > I agree that while we are discussing these points, we could also be > discussing how we solve problems of existing partitioning implementation > using whatever the above things end up being. Proposed approaches to > solve those problems might be usef

Re: [HACKERS] Parallel Seq Scan

2015-01-14 Thread Jim Nasby
On 1/13/15 9:42 PM, Amit Kapila wrote: As an example one of the the strategy could be if the table size is X MB and there are 8 workers, then divide the work as X/8 MB for each worker (which I have currently used in patch) and another could be each worker does scan 1 block at a time and then chec

Re: [HACKERS] Typo fix in alter_table.sgml

2015-01-14 Thread Michael Paquier
Robert Haas wrote: > But that's not a "typo" as stated in $SUBJECT but rather a "markup fix". Definitely. Thanks. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Peter Geoghegan
On Wed, Jan 14, 2015 at 4:53 PM, Merlin Moncure wrote: > yeah. via: > cds2=# \copy (select s as page, (bt_page_items('pg_class_oid_index', > s)).* from generate_series(1,12) s) to '/tmp/page_items.csv' csv > header; My immediate observation here is that blocks 2 and 9 have identical metadata (fr

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-14 Thread Josh Berkus
On 01/15/2015 02:24 AM, Robert Haas wrote: > I think your idea of adding read-only GUCs with the same names as all > of the recovey.conf parameters is a clear win. Even if we do nothing > more than that, it makes the values visible from the SQL level, and > that's good. But I think we should go f

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Merlin Moncure
On Wed, Jan 14, 2015 at 6:50 PM, Peter Geoghegan wrote: > This is great, but it's not exactly clear which bt_page_items() page > is which - some are skipped, but I can't be sure which. Would you mind > rewriting that query to indicate which block is under consideration by > bt_page_items()? yeah.

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Peter Geoghegan
This is great, but it's not exactly clear which bt_page_items() page is which - some are skipped, but I can't be sure which. Would you mind rewriting that query to indicate which block is under consideration by bt_page_items()? Thanks -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (

Re: [HACKERS] Shouldn't CREATE TABLE LIKE copy the relhasoids property?

2015-01-14 Thread Tom Lane
Andrew Dunstan writes: > On 01/14/2015 07:29 PM, Tom Lane wrote: >> If you don't find that problematic, how about this case? >> >> regression=# create table src2 (f1 int, primary key(oid)) with oids; >> CREATE TABLE >> regression=# create table dst2 (like src2 including indexes); >> ERROR: colum

Re: [HACKERS] Shouldn't CREATE TABLE LIKE copy the relhasoids property?

2015-01-14 Thread Andrew Dunstan
On 01/14/2015 07:29 PM, Tom Lane wrote: dst1 doesn't get an OID column: regression=# create table src1 (f1 int) with oids; CREATE TABLE regression=# create table dst1 (like src1); CREATE TABLE regression=# \d+ src1 Table "public.src1" Column | Type | Modifiers | S

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Merlin Moncure
On Wed, Jan 14, 2015 at 6:26 PM, Merlin Moncure wrote: > On Wed, Jan 14, 2015 at 5:39 PM, Peter Geoghegan wrote: >> On Wed, Jan 14, 2015 at 3:38 PM, Merlin Moncure wrote: >>> (gdb) print BufferGetBlockNumber(buf) >>> $15 = 9 >>> >>> ..and it stays 9, continuing several times having set breakpoi

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Peter Geoghegan
On Wed, Jan 14, 2015 at 4:26 PM, Merlin Moncure wrote: > The index is the oid index on pg_class. Some more info: > > *) temp table churn is fairly high. Several dozen get spawned and > destroted at the start of a replication run, all at once, due to some > dodgy coding via dblink. During the re

Re: [HACKERS] s_lock.h default definitions are rather confused

2015-01-14 Thread Tom Lane
Andres Freund writes: > Right now I think a #ifdef/undef S_UNLOCK in the relevant gcc section > sufficient and acceptable. It's after all the HPPA section that doesn't > really play by the rules. Works for me. regards, tom lane -- Sent via pgsql-hackers mailing list (p

[HACKERS] Shouldn't CREATE TABLE LIKE copy the relhasoids property?

2015-01-14 Thread Tom Lane
dst1 doesn't get an OID column: regression=# create table src1 (f1 int) with oids; CREATE TABLE regression=# create table dst1 (like src1); CREATE TABLE regression=# \d+ src1 Table "public.src1" Column | Type | Modifiers | Storage | Stats target | Description

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Merlin Moncure
On Wed, Jan 14, 2015 at 5:39 PM, Peter Geoghegan wrote: > On Wed, Jan 14, 2015 at 3:38 PM, Merlin Moncure wrote: >> (gdb) print BufferGetBlockNumber(buf) >> $15 = 9 >> >> ..and it stays 9, continuing several times having set breakpoint. > > > And the index involved? I'm pretty sure that this in

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Peter Geoghegan
On Wed, Jan 14, 2015 at 3:38 PM, Merlin Moncure wrote: > (gdb) print BufferGetBlockNumber(buf) > $15 = 9 > > ..and it stays 9, continuing several times having set breakpoint. And the index involved? I'm pretty sure that this in an internal page, no? -- Peter Geoghegan -- Sent via pgsql-hac

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Merlin Moncure
On Wed, Jan 14, 2015 at 2:32 PM, Peter Geoghegan wrote: > On Wed, Jan 14, 2015 at 12:24 PM, Peter Geoghegan wrote: >> Could you write some code to print out the block number (i.e. >> "BlockNumber blkno") if there are more than, say, 5 retries within >> _bt_moveright()? > > Obviously I mean that t

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-01-14 Thread Kouhei Kaigai
> On Fri, Jan 9, 2015 at 10:51 AM, Kouhei Kaigai wrote: > > When custom-scan node replaced a join-plan, it shall have at least two > > child plan-nodes. The callback handler of PlanCustomPath needs to be > > able to call create_plan_recurse() to transform the underlying > > path-nodes to plan-node

Re: [HACKERS] s_lock.h default definitions are rather confused

2015-01-14 Thread Andres Freund
On 2015-01-14 12:27:42 -0500, Robert Haas wrote: > On Mon, Jan 12, 2015 at 12:57 PM, Andres Freund > wrote: > > On 2015-01-10 23:03:36 +0100, Andres Freund wrote: > >> On 2015-01-10 16:09:42 -0500, Tom Lane wrote: > >> > I've not tried to build HEAD on my HPPA dinosaur for awhile, but I did > >>

Re: [HACKERS] __attribute__ for non-gcc compilers

2015-01-14 Thread Andres Freund
On 2015-01-14 17:46:39 -0500, Robert Haas wrote: > On Wed, Jan 14, 2015 at 5:29 PM, Andres Freund wrote: > > I think it's better than the alternatives: > > > > a) Don't support 64bit atomics on any 32bit platform. I think that'd be > >sad because there's some places that could greatly benefit

Re: [HACKERS] __attribute__ for non-gcc compilers

2015-01-14 Thread Robert Haas
On Wed, Jan 14, 2015 at 5:29 PM, Andres Freund wrote: > I think it's better than the alternatives: > > a) Don't support 64bit atomics on any 32bit platform. I think that'd be >sad because there's some places that could greatly benefit from being >able to read/store/manipulate e.g. LSNs ato

Re: [HACKERS] __attribute__ for non-gcc compilers

2015-01-14 Thread Andres Freund
On 2015-01-14 15:48:47 -0500, Robert Haas wrote: > On Tue, Jan 13, 2015 at 4:18 PM, Oskari Saarenmaa wrote: > > Commit db4ec2ffce35 added alignment attributes for 64-bit atomic > > variables as required on 32-bit platforms using > > __attribute__((aligned(8)). That works fine with GCC, and would

Re: [HACKERS] Out-of-bounds write and incorrect detection of trigger file in pg_standby

2015-01-14 Thread Robert Haas
On Wed, Jan 14, 2015 at 8:24 AM, Michael Paquier wrote: > In pg_standby.c, we use a 32-byte buffer in CheckForExternalTrigger to > which is read the content of the trigger file defined by -f: > CheckForExternalTrigger(void) > { > charbuf[32]; > [...] > if ((len = read(fd, buf,

Re: [HACKERS] pg_rewind in contrib

2015-01-14 Thread Robert Haas
On Wed, Jan 7, 2015 at 8:44 AM, Andrew Dunstan wrote: > I understand, but I think "pg_rewind" is likely to be misleading to many > users who will say "but I don't want just to rewind". > > I'm not wedded to the name I suggested, but I think we should look at > possible alternative names. We do hav

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2015-01-14 Thread Tomas Vondra
On 12.1.2015 01:28, Ali Akbar wrote: > > > Or else we implement what you suggest below (more comments below): > > > > Thinking about the 'release' flag a bit more - maybe we > could do > > this > > instead: > > > > if (release

Re: [HACKERS] orangutan seizes up during isolation-check

2015-01-14 Thread Peter Eisentraut
On 1/1/15 11:04 PM, Noah Misch wrote: >> Clusters hosted on OS X fall into these categories: >> >> 1) Unaffected configuration. This includes everyone setting a valid messages >>locale via LANG, LC_ALL or LC_MESSAGES. >> 2) Affected configuration. Through luck and light use, the cluster would

Re: [HACKERS] Async execution of postgres_fdw.

2015-01-14 Thread Robert Haas
On Tue, Jan 13, 2015 at 6:46 AM, Kyotaro HORIGUCHI wrote: >> Is it possible to use the parallel query infrastructure being built by >> Robert or to do something like parallel seq scan? That will work, not just >> for Postgres FDW but all the FDWs. > > But, I think, from the performance view, every

Re: [HACKERS] Improving RLS qual pushdown

2015-01-14 Thread Robert Haas
On Wed, Jan 14, 2015 at 9:22 AM, Dean Rasheed wrote: > On 14 January 2015 at 13:29, Robert Haas wrote: >> One thing they could still leak is the number of times they got >> called, and thus possibly the number of unseen rows. Now if the >> expressions get constant-folded away that won't be an is

Re: [HACKERS] Typo fix in alter_table.sgml

2015-01-14 Thread Robert Haas
On Wed, Jan 14, 2015 at 2:55 AM, Michael Paquier wrote: > I noticed that SET STATISTICS was not in a block in > alter_table.sgml: > > - SET STATISTICS acquires a SHARE UPDATE EXCLUSIVE > lock. > + SET STATISTICS acquires a > + SHARE UPDATE EXCLUSIVE lock. > > That's

Re: [HACKERS] Parallel Seq Scan

2015-01-14 Thread Robert Haas
On Tue, Jan 13, 2015 at 6:25 AM, John Gorman wrote: > One approach that I has worked well for me is to break big jobs into much > smaller bite size tasks. Each task is small enough to complete quickly. > > We add the tasks to a task queue and spawn a generic worker pool which eats > through the ta

Re: [HACKERS] OOM on EXPLAIN with lots of nodes

2015-01-14 Thread Robert Haas
On Tue, Jan 13, 2015 at 8:16 PM, Tom Lane wrote: > I wrote: >> Heikki Linnakangas writes: >>> But do we really need to backpatch any of this? > >> Alexey's example consumes only a couple hundred MB in 9.2, vs about 7GB >> peak in 9.3 and up. That seems like a pretty nasty regression. > > I did a

Re: [HACKERS] Safe memory allocation functions

2015-01-14 Thread Robert Haas
On Tue, Jan 13, 2015 at 10:10 AM, Tom Lane wrote: > However, there is a larger practical problem with this whole concept, > which is that experience should teach us to be very wary of the assumption > that asking for memory the system can't give us will just lead to nice > neat malloc-returns-NULL

Re: [HACKERS] __attribute__ for non-gcc compilers

2015-01-14 Thread Robert Haas
On Tue, Jan 13, 2015 at 4:18 PM, Oskari Saarenmaa wrote: > Commit db4ec2ffce35 added alignment attributes for 64-bit atomic > variables as required on 32-bit platforms using > __attribute__((aligned(8)). That works fine with GCC, and would work > with Solaris Studio Compiler [1] and IBM XL C [2],

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Peter Geoghegan
On Wed, Jan 14, 2015 at 12:24 PM, Peter Geoghegan wrote: > Could you write some code to print out the block number (i.e. > "BlockNumber blkno") if there are more than, say, 5 retries within > _bt_moveright()? Obviously I mean that the block number should be printed, no matter whether or not the P

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Peter Geoghegan
On Wed, Jan 14, 2015 at 11:49 AM, Merlin Moncure wrote: > so it looks like nobody ever exits from _bt_moveright. any last > requests before I start bisecting down? Could you write some code to print out the block number (i.e. "BlockNumber blkno") if there are more than, say, 5 retries within _b

Re: [HACKERS] Memory leak in vacuumlo

2015-01-14 Thread Robert Haas
On Tue, Jan 13, 2015 at 2:58 AM, Michael Paquier wrote: > Coverity has pointed out that vacuumlo.c is leaking memory when > grabbing the list of candidate table names and schemas. Attached is a > patch to fix the leak. It's only leaking in error cases, the first of which is quite unlikely. But I

Re: [HACKERS] PATCH: hashjoin - gracefully increasing NTUP_PER_BUCKET instead of batching

2015-01-14 Thread Tomas Vondra
On 11.12.2014 23:46, Tomas Vondra wrote: > On 11.12.2014 22:16, Robert Haas wrote: >> On Thu, Dec 11, 2014 at 2:51 PM, Tomas Vondra wrote: >> >>> The idea was that if we could increase the load a bit (e.g. using 2 >>> tuples per bucket instead of 1), we will still use a single batch in >>> some ca

Re: [HACKERS] [PATCH] explain sortorder

2015-01-14 Thread Arne Scheffer
Hi, we will also remove the following is lc_collate hint in the next version, showing only mandatory info as suggested. /* for those who use COLLATE although their default is already the wanted */ if (strcmp(collname, localeptr) == 0) {

Re: [HACKERS] [PATCH] explain sortorder

2015-01-14 Thread Timmer, Marius
Hello Heikki, abbreviated version: Sorry, the problem is only the unhandy patch text format, not different opinions how to proceed. Long version: The v7 patch file already addressed your suggestions, but the file contained serveral (old) local commits, the new ones at the end of the patch text/f

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Merlin Moncure
On Wed, Jan 14, 2015 at 9:49 AM, Andres Freund wrote: > On 2015-01-14 09:47:19 -0600, Merlin Moncure wrote: >> On Wed, Jan 14, 2015 at 9:30 AM, Andres Freund >> wrote: >> > If you gdb in, and type 'fin' a couple times, to wait till the function >> > finishes, is there actually any progress? I'm

Re: [HACKERS] pg_basebackup fails with long tablespace paths

2015-01-14 Thread Robert Haas
On Tue, Jan 13, 2015 at 4:41 PM, Peter Eisentraut wrote: > I think the key point I'm approaching is that the information should > only ever be in one place, all the time. This is not dissimilar from > why we took the tablespace location out of the system catalogs. Users > might have all kinds of

Re: [HACKERS] Removing INNER JOINs

2015-01-14 Thread Jim Nasby
On 1/13/15 5:02 AM, David Rowley wrote: I can't quite get my head around what you mean here, as the idea sounds quite similar to something that's been discussed already and ruled out. If we're joining relation a to relation b, say the plan chosen is a merge join. If we put some special node as

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-14 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: > Turns out it wasn't as simple as that. prepend_row_security_policies() > really could get called multiple times for the same RTE, because the > call to query_tree_walker() at the end of fireRIRrules() would descend > into the just-added quals again

Re: [HACKERS] ereport bug

2015-01-14 Thread Robert Haas
On Wed, Jan 14, 2015 at 1:40 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Jan 12, 2015 at 6:27 AM, Dmitry Voronin >> wrote: >>> I am attaching to this letter a test case that shows the behavior >>> errcontext() macro and the way to fix it. > >> So the upshot of this is that given errfini

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-14 Thread Dean Rasheed
On 14 January 2015 at 08:43, Dean Rasheed wrote: > On 12 January 2015 at 14:24, Stephen Frost wrote: >> Interesting, thanks for the work! I had been suspicious that there was >> an issue with the recursion handling. >> > > So continuing to review the RLS code, I spotted the following in > prepen

Re: [HACKERS] libpq bad async behaviour

2015-01-14 Thread Daurnimator
On 14 January 2015 at 08:40, Andres Freund wrote: > I think that kind of solution isn't likely to be satisfying. The amount > of porting work is just not going to be worth the cost. And it won't be > easily hideable in the API at all as the callers will expect a normal > fd. > All that consumers

Re: [HACKERS] ereport bug

2015-01-14 Thread Tom Lane
Robert Haas writes: > On Mon, Jan 12, 2015 at 6:27 AM, Dmitry Voronin > wrote: >> I am attaching to this letter a test case that shows the behavior >> errcontext() macro and the way to fix it. > So the upshot of this is that given errfinish(A, B, C), where A, B, > and C are expressions, my gcc i

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Peter Geoghegan
On Wed, Jan 14, 2015 at 7:22 AM, Merlin Moncure wrote: > I'll try to pull commits that Peter suggested and see if that helps > (I'm getting ready to bring the database down). I can send the code > off-list if you guys think it'd help. Thanks for the code! I think it would be interesting to see

Re: [HACKERS] s_lock.h default definitions are rather confused

2015-01-14 Thread Robert Haas
On Mon, Jan 12, 2015 at 12:57 PM, Andres Freund wrote: > On 2015-01-10 23:03:36 +0100, Andres Freund wrote: >> On 2015-01-10 16:09:42 -0500, Tom Lane wrote: >> > I've not tried to build HEAD on my HPPA dinosaur for awhile, but I did >> > just now, and I am presented with boatloads of this: >> > >>

Re: [HACKERS] ereport bug

2015-01-14 Thread Robert Haas
On Mon, Jan 12, 2015 at 6:27 AM, Dmitry Voronin wrote: > I am attaching to this letter a test case that shows the behavior > errcontext() macro and the way to fix it. So the upshot of this is that given errfinish(A, B, C), where A, B, and C are expressions, my gcc is choosing to evaluate C, then

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-01-14 Thread Robert Haas
On Fri, Jan 9, 2015 at 10:51 AM, Kouhei Kaigai wrote: > When custom-scan node replaced a join-plan, it shall have at least two > child plan-nodes. The callback handler of PlanCustomPath needs to be > able to call create_plan_recurse() to transform the underlying path-nodes > to plan-nodes, because

Re: [HACKERS] Merging postgresql.conf and postgresql.auto.conf

2015-01-14 Thread Glyn Astill
- Original Message - > From: Sawada Masahiko > To: Tom Lane > Cc: PostgreSQL-development > Sent: Wednesday, 14 January 2015, 16:09 > Subject: Re: [HACKERS] Merging postgresql.conf and postgresql.auto.conf > > On Thu, Jan 15, 2015 at 12:37 AM, Tom Lane wrote: >> Sawada Masahiko writ

Re: [HACKERS] Compression of full-page-writes

2015-01-14 Thread Robert Haas
On Fri, Jan 2, 2015 at 11:52 AM, Bruce Momjian wrote: > OK, so given your stats, the feature give a 12.5% reduction in I/O. If > that is significant, shouldn't we see a performance improvement? If we > don't see a performance improvement, is I/O reduction worthwhile? Is it > valuable in that it

Re: [HACKERS] Re: Patch to add functionality to specify ORDER BY in CREATE FUNCTION for SRFs

2015-01-14 Thread Robert Haas
On Mon, Jan 5, 2015 at 12:54 PM, Tom Lane wrote: > TBH, my first reaction to this entire patch is unfavorable: it's a > solution in search of a problem. It adds substantial complication not > only for users but for PG developers in order to solve a rather narrow > performance issue. > > What woul

Re: [HACKERS] Merging postgresql.conf and postgresql.auto.conf

2015-01-14 Thread Sawada Masahiko
On Thu, Jan 15, 2015 at 1:15 AM, Tom Lane wrote: > Sawada Masahiko writes: >> On Thu, Jan 15, 2015 at 12:37 AM, Tom Lane wrote: >>> Yes. The entire reason that postgresql.auto.conf is separate is that >>> we despaired of reading and rewriting postgresql.conf automatically >>> without making a h

Re: [HACKERS] [RFC] Incremental backup v3: incremental PoC

2015-01-14 Thread Gabriele Bartolini
Hi Marco, thank you for sending an updated patch. I am writing down a report of this initial (and partial) review. IMPORTANT: This patch is not complete, as stated by Marco. See the "Conclusions" section for my proposed TODO list. == Patch application I have been able to successfully apply yo

Re: [HACKERS] Merging postgresql.conf and postgresql.auto.conf

2015-01-14 Thread Tom Lane
Sawada Masahiko writes: > On Thu, Jan 15, 2015 at 12:37 AM, Tom Lane wrote: >> Yes. The entire reason that postgresql.auto.conf is separate is that >> we despaired of reading and rewriting postgresql.conf automatically >> without making a hash of material in the comments. Calling the logic >> a

Re: [HACKERS] Minor configure tweak to simplify adjusting gcc warnings

2015-01-14 Thread Andres Freund
On 2015-01-14 11:09:10 -0500, Tom Lane wrote: > Andres Freund writes: > > I've already given up... Given how infrequent it is, suppressing it for > > gull seems sufficient. > > I'm confused --- I see no format warnings in gull's current reports. Sorry it was me being confused. I somehow switched

Re: [HACKERS] Minor configure tweak to simplify adjusting gcc warnings

2015-01-14 Thread Tom Lane
Andres Freund writes: > I've already given up... Given how infrequent it is, suppressing it for > gull seems sufficient. I'm confused --- I see no format warnings in gull's current reports. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

Re: [HACKERS] Merging postgresql.conf and postgresql.auto.conf

2015-01-14 Thread Sawada Masahiko
On Thu, Jan 15, 2015 at 12:37 AM, Tom Lane wrote: > Sawada Masahiko writes: >> The postgresql.auto.conf is loaded after loading of postgresql.conf >> whenever configuration file is loaded or reloaded. >> This means that parameter in postgresql.auto.conf is quite high >> priority, so the parameter

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Andres Freund
On 2015-01-14 09:47:19 -0600, Merlin Moncure wrote: > On Wed, Jan 14, 2015 at 9:30 AM, Andres Freund wrote: > > If you gdb in, and type 'fin' a couple times, to wait till the function > > finishes, is there actually any progress? I'm wondering whether it's > > just many catalog accesses + contenti

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Merlin Moncure
On Wed, Jan 14, 2015 at 9:30 AM, Andres Freund wrote: > If you gdb in, and type 'fin' a couple times, to wait till the function > finishes, is there actually any progress? I'm wondering whether it's > just many catalog accesses + contention, or some other > problem. Alternatively set a breakpoint

Re: [HACKERS] Merging postgresql.conf and postgresql.auto.conf

2015-01-14 Thread Tom Lane
Sawada Masahiko writes: > The postgresql.auto.conf is loaded after loading of postgresql.conf > whenever configuration file is loaded or reloaded. > This means that parameter in postgresql.auto.conf is quite high > priority, so the parameter in postgresql.conf does not work at all > even if user s

[HACKERS] Merging postgresql.conf and postgresql.auto.conf

2015-01-14 Thread Sawada Masahiko
Hi all, The postgresql.auto.conf is loaded after loading of postgresql.conf whenever configuration file is loaded or reloaded. This means that parameter in postgresql.auto.conf is quite high priority, so the parameter in postgresql.conf does not work at all even if user set it manually. If user w

Re: [HACKERS] [PATCH] explain sortorder

2015-01-14 Thread Heikki Linnakangas
On 01/14/2015 05:26 PM, Timmer, Marius wrote: Hello Heikki, abbreviated version: Sorry, the problem is only the unhandy patch text format, not different opinions how to proceed. Long version: The v7 patch file already addressed your suggestions, but the file contained serveral (old) local comm

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Andres Freund
On 2015-01-14 09:22:45 -0600, Merlin Moncure wrote: > On Wed, Jan 14, 2015 at 9:11 AM, Andres Freund wrote: > > On 2015-01-14 10:05:01 -0500, Tom Lane wrote: > >> Merlin Moncure writes: > >> > On Wed, Jan 14, 2015 at 8:41 AM, Tom Lane wrote: > >> >> What are the autovac processes doing (accordin

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Merlin Moncure
On Wed, Jan 14, 2015 at 9:11 AM, Andres Freund wrote: > On 2015-01-14 10:05:01 -0500, Tom Lane wrote: >> Merlin Moncure writes: >> > On Wed, Jan 14, 2015 at 8:41 AM, Tom Lane wrote: >> >> What are the autovac processes doing (according to pg_stat_activity)? >> >> > pid,running,waiting,query >> >

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-14 Thread Andres Freund
On 2015-01-14 10:13:32 -0500, Tom Lane wrote: > Merlin Moncure writes: > > Yes, it is pg_class is coming from LockBufferForCleanup (). As you > > can see above, it has a shorter runtime. So it was killed off once > > about a half hour ago which did not free up the logjam. However, AV > > spaw

  1   2   >