[HACKERS] pg_ctl/pg_rewind tests vs. slow AIX buildfarm members

My AIX buildfarm members have failed the BinInstallCheck step on and off since inception. It became more frequent when I added animals sungazer and tern alongside the older hornet and mandrill. The animals share a machine with each other and with dozens of other developers. I setpriority() the a

Re: [HACKERS] Horizontal scalability/sharding

>> One lesson from XL we got is that we need testing framework for cluster, >> so any cluster project should at least pass functional and performance >> testing. >> > > +1. In early XC days, we focused a lot on adding newer features and > supporting as many PG features as possible. That took its t

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 9:04 PM, Oleg Bartunov wrote: > > > > One lesson from XL we got is that we need testing framework for cluster, > so any cluster project should at least pass functional and performance > testing. > +1. In early XC days, we focused a lot on adding newer features and supporti

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/09/03 9:41, Robert Haas wrote: That having been said, I don't entirely like Fujita-san's patch either. Much of the new code is called immediately adjacent to an FDW callback which could pretty trivially do the same thing itself, if needed. Another idea about that code is to call that c

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On Wed, Sep 2, 2015 at 9:43 PM, David Rowley wrote: > Peter, would you be able to share the test case which you saw the speedup > on. So far I've been unable to see much of an improvement. The case I tested was an internal sort CREATE INDEX. I don't recall the exact details, but testing showed it

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On 3 September 2015 at 07:24, Peter Geoghegan wrote: > On Wed, Sep 2, 2015 at 7:12 AM, Andres Freund wrote: > > What worries me about adding explicit prefetching is that it's *highly* > > platform and even micro-architecture dependent. Why is looking three > > elements ahead the right value? > >

Re: [HACKERS] WIP: Make timestamptz_out less slow.

On 3 September 2015 at 05:10, Andres Freund wrote: > On 2015-08-09 12:47:53 +1200, David Rowley wrote: > > I took a bit of weekend time to finish this one off. Patch attached. > > > > A quick test shows a pretty good performance increase: > > > > create table ts (ts timestamp not null); > > inser

Re: [HACKERS] GIN pending clean up is not interruptable

On Thu, Sep 3, 2015 at 2:15 AM, Andres Freund wrote: > On 2015-08-12 11:59:48 -0700, Jeff Janes wrote: >> Attached patch does it that way. There was also a free-standing >> CHECK_FOR_INTERRUPTS() which had no reason that I could see not be a >> vacuum_delay_point, so I changed that one as well.

Re: [HACKERS] Horizontal scalability/sharding

On Thu, Sep 3, 2015 at 8:28 AM, Bruce Momjian wrote: > > On Wed, Sep 2, 2015 at 07:50:25PM -0700, Joshua Drake wrote: > > >Can you explain why logical replication is better than binary > > >replication for this use-case? > > > > > > > Selectivity? > > I was assuming you would just create identica

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 09:03:25PM -0400, Robert Haas wrote: > > Can you explain why logical replication is better than binary > > replication for this use-case? > > Uh, well, for the same reasons it is better in many other cases. > Particularly, you probably don't want to replicate all the data o

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 07:50:25PM -0700, Joshua Drake wrote: > >Can you explain why logical replication is better than binary > >replication for this use-case? > > > > Selectivity? I was assuming you would just create identical slaves to handle failure, rather than moving selected data around.

Re: [HACKERS] Horizontal scalability/sharding

On 09/02/2015 03:56 PM, Bruce Momjian wrote: On Wed, Sep 2, 2015 at 02:41:46PM -0400, Robert Haas wrote: 4. Therefore, I think that we should instead use logical replication, which might be either synchronous or asynchronous. When you modify one copy of the data, that change will then be repli

Re: [HACKERS] Allow replication roles to use file access functions

On Thu, Sep 3, 2015 at 11:20 AM, Stephen Frost wrote: > * Michael Paquier wrote: >> 1) Use a differential backup to me, or the possibility to fetch a set >> of data block diffs from a source node using an LSN and then re-apply >> them on the target node. The major disadvantage of this approach is >

Re: [HACKERS] pg_stat_statements query jumbling question

On 2015/09/01 14:39, Satoshi Nagayasu wrote: > On 2015/09/01 14:01, Tom Lane wrote: >> Satoshi Nagayasu writes: >>> On 2015/09/01 13:41, Peter Geoghegan wrote: If you want to use the queryId field directly, which I recall you mentioning before, then that's harder. There is simply no cont

Re: [HACKERS] Horizontal scalability/sharding

On Thu, Sep 3, 2015 at 3:41 AM, Robert Haas wrote: > 3. IIUC, Postgres-XC handles this problem by reducing at least > volatile functions, maybe all functions, to constants. Then it > generates an SQL statement to be sent to the data node to make the > appropriate change. If there's more than one

Re: [HACKERS] Allow replication roles to use file access functions

* Michael Paquier (michael.paqu...@gmail.com) wrote: > On Thu, Sep 3, 2015 at 10:05 AM, Stephen Frost wrote: > > * Michael Paquier (michael.paqu...@gmail.com) wrote: > >> On Thu, Sep 3, 2015 at 4:52 AM, Alvaro Herrera wrote: > > The replication role already has read-only access to everything > > (n

Re: [HACKERS] Allow replication roles to use file access functions

On Thu, Sep 3, 2015 at 10:05 AM, Stephen Frost wrote: > * Michael Paquier (michael.paqu...@gmail.com) wrote: >> On Thu, Sep 3, 2015 at 4:52 AM, Alvaro Herrera wrote: > The replication role already has read-only access to everything > (nearly?) in the PGDATA directory. The specific issue in this ca

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On 9/2/15 2:56 PM, Jim Nasby wrote: On 9/2/15 2:17 PM, Alvaro Herrera wrote: Michael Paquier wrote: I haven't written yet a test case but I think that we could reproduce that simply by having a relation referenced in the exception block of a first function, calling a second function that itsel

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

That doesn't match any of the empirical tests I did at the time. I posted graphs of the throughput for varying numbers of spindles with varying amount of prefetch. In every case more prefetching increases throuput up to N times the single platter throuput where N was the number of spindles. There

Re: [HACKERS] Improving test coverage of extensions with pg_dump

On Thu, Sep 3, 2015 at 12:38 AM, Andres Freund wrote: > Isn't a full test with a separate initdb, create extension etc. a really > heavyhanded way to test this? I mean that's a test where the setup takes > up to 10s, whereas the actual runtime is in the millisecond range? > > Adding tests in this m

Re: [HACKERS] Improving test coverage of extensions with pg_dump

On Thu, Sep 3, 2015 at 2:30 AM, Alvaro Herrera wrote: > Andres Freund wrote: >> Isn't a full test with a separate initdb, create extension etc. a really >> heavyhanded way to test this? I mean that's a test where the setup takes >> up to 10s, whereas the actual runtime is in the millisecond range?

Re: [HACKERS] Allow replication roles to use file access functions

Micahel, * Michael Paquier (michael.paqu...@gmail.com) wrote: > On Thu, Sep 3, 2015 at 4:52 AM, Alvaro Herrera > wrote: > > Michael Paquier wrote: > >> As of now, file access functions in genfile.c can only be used by > >> superusers. This proposal is to relax those functions so as > >> replicat

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 6:56 PM, Bruce Momjian wrote: > On Wed, Sep 2, 2015 at 02:41:46PM -0400, Robert Haas wrote: >> 4. Therefore, I think that we should instead use logical replication, >> which might be either synchronous or asynchronous. When you modify >> one copy of the data, that change w

Re: [HACKERS] src/test/ssl broken on HEAD

On Thu, Sep 3, 2015 at 5:22 AM, Robert Haas wrote: > Still, that's not a reason not commit this, so done. Thanks. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pgbench stats per script & other stuff

On Wed, Sep 2, 2015 at 5:55 PM, Andres Freund wrote: > On 2015-09-02 14:36:51 -0400, Robert Haas wrote: >> On Wed, Sep 2, 2015 at 2:20 PM, Fabien COELHO wrote: >> >> I'm wondering if percentages instead of weights would be a better >> >> idea. That'd mean you'd be forced to be more careful when a

Re: [HACKERS] exposing pg_controldata and pg_config as functions

On Wed, Sep 2, 2015 at 5:31 PM, Joe Conway wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 09/02/2015 05:25 PM, Tom Lane wrote: >> Robert Haas writes: >>> But I'm not sure I like the idea of adding a server dependency on >>> the ability to exec pg_controldata. That seems like it

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

, On Wed, Aug 26, 2015 at 4:05 AM, Kouhei Kaigai wrote: >> On 2015/08/26 16:07, Kouhei Kaigai wrote: >> I wrote: >> >> Maybe I'm missing something, but why do we need such a flexiblity for >> >> the columnar-stores? >> >> > Even if we enforce them a new interface specification comfortable to RDBMS

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On 2015-09-02 17:14:12 -0700, Peter Geoghegan wrote: > On Wed, Sep 2, 2015 at 12:24 PM, Peter Geoghegan wrote: > > On Wed, Sep 2, 2015 at 7:12 AM, Andres Freund wrote: > >> On 2015-07-19 16:34:52 -0700, Peter Geoghegan wrote: > >> Hm. Is a compiler test actually test anything reliably here? Won't

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On 2015-09-03 01:59:13 +0200, Tomas Vondra wrote: > That's a bit surprising, especially considering that e_i_c=30 means ~100 > pages to prefetch if I'm doing the math right. > > AFAIK queue depth for SATA drives generally is 32 (so prefetching 100 pages > should not make a difference), 256 for SAS

Re: [HACKERS] Allow replication roles to use file access functions

On Thu, Sep 3, 2015 at 8:59 AM, Andres Freund wrote: > On 2015-09-02 19:48:15 -0400, Tom Lane wrote: >> Just on general principles, this seems like a pretty horrid idea. >> To me replication privilege means the ability to transfer data out of >> the master, not to cause arbitrary state changes on

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On Wed, Sep 2, 2015 at 12:24 PM, Peter Geoghegan wrote: > On Wed, Sep 2, 2015 at 7:12 AM, Andres Freund wrote: >> On 2015-07-19 16:34:52 -0700, Peter Geoghegan wrote: >> Hm. Is a compiler test actually test anything reliably here? Won't this >> just throw a warning during compile time about an un

Re: [HACKERS] Allow replication roles to use file access functions

On 2015-09-02 19:48:15 -0400, Tom Lane wrote: > Just on general principles, this seems like a pretty horrid idea. > To me replication privilege means the ability to transfer data out of > the master, not to cause arbitrary state changes on the master. It's not about the permission to trigger pg_re

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On 09/03/2015 12:23 AM, Andres Freund wrote: On 2015-09-02 14:31:35 -0700, Josh Berkus wrote: On 09/02/2015 02:25 PM, Tomas Vondra wrote: As I explained, spindles have very little to do with it - you need multiple I/O requests per device, to get the benefit. Sure, the DBAs should know how ma

Re: [HACKERS] Allow replication roles to use file access functions

Michael Paquier writes: > On Thu, Sep 3, 2015 at 4:52 AM, Alvaro Herrera > wrote: >> Michael Paquier wrote: >>> The main argument for this change is that pg_rewind makes use of those >>> functions, forcing users to use a superuser role when rewinding a >>> node. >> Can this piggyback on Stephen

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On Wed, Sep 2, 2015 at 4:26 PM, Andres Freund wrote: > I'm not following? Just write pg_read_prefetch(state->memtuples + 3 + > readptr->current) and the corresponding version for tuplesort in the > callsites? Oh, I see. Maybe I'll do it that way when I pick this up in a little while. I need to co

Re: [HACKERS] Allow replication roles to use file access functions

On Thu, Sep 3, 2015 at 4:52 AM, Alvaro Herrera wrote: > Michael Paquier wrote: >> Hi all, >> >> As of now, file access functions in genfile.c can only be used by >> superusers. This proposal is to relax those functions so as >> replication users can use them as well. Here are the functions aimed >

Re: [HACKERS] Allow replication roles to use file access functions

On Wed, Sep 2, 2015 at 11:32 PM, Fujii Masao wrote: > Did you confirm that replication user can complete pg_rewind > after this patch is applied? Yes. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.or

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On 2015-09-02 16:23:13 -0700, Peter Geoghegan wrote: > On Wed, Sep 2, 2015 at 4:12 PM, Andres Freund wrote: > >> Well, still needs to work for tuplestore, which does not have a SortTuple. > > > > Isn't it even more trivial there? It's just an array of void*'s? So > > prefetch(state->memtuples + 3

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On Wed, Sep 2, 2015 at 4:12 PM, Andres Freund wrote: >> Well, still needs to work for tuplestore, which does not have a SortTuple. > > Isn't it even more trivial there? It's just an array of void*'s? So > prefetch(state->memtuples + 3 + readptr->current)? All I meant is that there couldn't be one

Re: [HACKERS] src/test/ssl broken on HEAD

On 2015-09-02 17:03:46 -0400, Robert Haas wrote: > On Wed, Sep 2, 2015 at 4:50 PM, Andrew Dunstan wrote: > > Tell me what's needed and I'll look at creating a buildfarm test module for > > it. > > You run the tests via: make -C src/test/ssl check > > But nota bene security caveats: > > commit e

Re: [HACKERS] DBT-3 with SF=20 got failed

Hello KaiGai-san, I've discovered a bug in the proposed patch - when resetting the hash table after the first batch, it does this: memset(hashtable->buckets, 0, sizeof(nbuckets * sizeof(HashJoinTuple))); The first 'sizeof' is bogus, so this only zeroes the first 8 bytes of the array (usually

Re: [HACKERS] Pg_upgrade remote copy

On Wed, Sep 2, 2015 at 04:50:32PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Fri, Aug 28, 2015 at 10:34:38PM -0700, AI Rumman wrote: > > > > In pg_upgrade, how about adding a feature to copy data directory over > > > network. > > > That is, we can run pg_upgrade from our new host

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 12:03:36PM -0700, Josh Berkus wrote: > Well, there is a WIP patch for that, which IMHO would be much improved > by having a concrete use-case like this one. What nobody is working on > -- and we've vetoed in the past -- is a way of automatically failing and > removing from

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On 2015-09-02 16:02:00 -0700, Peter Geoghegan wrote: > On Wed, Sep 2, 2015 at 3:13 PM, Andres Freund wrote: > > That's just a question of how to formulate this though? > > > > pg_rfetch(((char *) state->memtuples ) + 3 * sizeof(SortTuple) + > > offsetof(SortTuple, tuple))? > > > > For something h

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On Wed, Sep 2, 2015 at 3:13 PM, Andres Freund wrote: > I'd be less brief in this case then, no need to be super short here. Okay. >> It started out that way, but Tom felt that it was better to have a >> USE_MEM_PREFETCH because of the branch below... > > That doesn't mean we shouldn't still prov

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 02:41:46PM -0400, Robert Haas wrote: > 4. Therefore, I think that we should instead use logical replication, > which might be either synchronous or asynchronous. When you modify > one copy of the data, that change will then be replicated to all other > nodes. If you are OK

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On 2015-09-02 19:49:13 +0100, Greg Stark wrote: > I can take the blame for this formula. > > It's called the "Coupon Collector Problem". If you hit get a random > coupon from a set of n possible coupons, how many random coupons would > you have to collect before you expect to have at least one of

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On 2015-09-02 14:31:35 -0700, Josh Berkus wrote: > On 09/02/2015 02:25 PM, Tomas Vondra wrote: > > > > As I explained, spindles have very little to do with it - you need > > multiple I/O requests per device, to get the benefit. Sure, the DBAs > > should know how many spindles they have and should

Re: [HACKERS] Improving replay of XLOG_BTREE_VACUUM records

On 2015-09-02 22:46:59 +0100, Greg Stark wrote: > On Sun, Jul 26, 2015 at 1:46 PM, Andres Freund wrote: > > To me it sounds like this shouldn't go through the full ReadBuffer() > > rigamarole. That code is already complex enough, and here it's really > > not needed. I think it'll be much easier to

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On 2015-09-02 12:24:33 -0700, Peter Geoghegan wrote: > "Read fetch". One argument past to the intrinsic here specifies that > the variable will be read only. I did things this way because I > imagined that there would be very limited uses for the macro only. I > probably cover almost all interestin

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On Wed, Sep 2, 2015 at 2:31 PM, Josh Berkus wrote: > On 09/02/2015 02:25 PM, Tomas Vondra wrote: > > > > As I explained, spindles have very little to do with it - you need > > multiple I/O requests per device, to get the benefit. Sure, the DBAs > > should know how many spindles they have and shou

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On Wed, Sep 2, 2015 at 4:31 PM, Josh Berkus wrote: > On 09/02/2015 02:25 PM, Tomas Vondra wrote: >> >> As I explained, spindles have very little to do with it - you need >> multiple I/O requests per device, to get the benefit. Sure, the DBAs >> should know how many spindles they have and should be

Re: [HACKERS] pgbench stats per script & other stuff

On 2015-09-02 14:36:51 -0400, Robert Haas wrote: > On Wed, Sep 2, 2015 at 2:20 PM, Fabien COELHO wrote: > >> I'm wondering if percentages instead of weights would be a better > >> idea. That'd mean you'd be forced to be more careful when adding another > >> script (having to adjust the percentages

Re: [HACKERS] exposing pg_controldata and pg_config as functions

Josh Berkus wrote: > On 09/02/2015 02:34 PM, Alvaro Herrera wrote: > > I think trying to duplicate the exact strings isn't too nice an > > interface. > > Well, for pg_controldata, no, but what else would you do for pg_config? I was primarily looking at pg_controldata, so we agree there. As for

Re: [HACKERS] Improving replay of XLOG_BTREE_VACUUM records

On Sun, Jul 26, 2015 at 1:46 PM, Andres Freund wrote: > To me it sounds like this shouldn't go through the full ReadBuffer() > rigamarole. That code is already complex enough, and here it's really > not needed. I think it'll be much easier to review - and actually faster > in many cases to simply

Re: [HACKERS] exposing pg_controldata and pg_config as functions

On 09/02/2015 02:34 PM, Alvaro Herrera wrote: > Tom Lane wrote: >> Robert Haas writes: >>> But I'm not sure I like the idea of adding a server dependency on the >>> ability to exec pg_controldata. That seems like it could be >>> unreliable at best, and a security vulnerability at worst. >> >> I h

Re: [HACKERS] Horizontal scalability/sharding

On 09/02/2015 12:30 PM, Robert Haas wrote: > On Wed, Sep 2, 2015 at 3:03 PM, Josh Berkus wrote: >>> 4. Therefore, I think that we should instead use logical replication, >>> which might be either synchronous or asynchronous. When you modify >>> one copy of the data, that change will then be repli

Re: [HACKERS] exposing pg_controldata and pg_config as functions

Tom Lane wrote: > Robert Haas writes: > > But I'm not sure I like the idea of adding a server dependency on the > > ability to exec pg_controldata. That seems like it could be > > unreliable at best, and a security vulnerability at worst. > > I hadn't been paying attention --- the proposed patch

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On 09/02/2015 02:25 PM, Tomas Vondra wrote: > > As I explained, spindles have very little to do with it - you need > multiple I/O requests per device, to get the benefit. Sure, the DBAs > should know how many spindles they have and should be able to determine > optimal IO depth. But we actually sa

Re: [HACKERS] exposing pg_controldata and pg_config as functions

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/02/2015 05:25 PM, Tom Lane wrote: > Robert Haas writes: >> But I'm not sure I like the idea of adding a server dependency on >> the ability to exec pg_controldata. That seems like it could be >> unreliable at best, and a security vulnerability

Re: [HACKERS] exposing pg_controldata and pg_config as functions

Robert Haas writes: > But I'm not sure I like the idea of adding a server dependency on the > ability to exec pg_controldata. That seems like it could be > unreliable at best, and a security vulnerability at worst. I hadn't been paying attention --- the proposed patch actually depends on exec'in

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

Hi, On 09/02/2015 08:49 PM, Greg Stark wrote: On 2 Sep 2015 14:54, "Andres Freund" wrote: + /*-- + * The user-visible GUC parameter is the number of drives (spindles), + * which we need to translate to a number-of-pages-to-prefetch target. + * The target value is

Re: [HACKERS] Improving replay of XLOG_BTREE_VACUUM records

25 авг. 2015 г., в 16:03, Michael Paquier написал(а):On Sun, Jul 26, 2015 at 9:46 PM, Andres Freund wrote:On 2015-07-24 09:53:49 +0300, Heikki Linnakangas wrote:To me it sounds like this shouldn't go through the full ReadBuffer()rigamarole. That code is already complex e

Re: [HACKERS] exposing pg_controldata and pg_config as functions

On Mon, Aug 31, 2015 at 9:47 PM, Peter Eisentraut wrote: > On 8/25/15 11:32 PM, Joe Conway wrote: >> 1.) pg_controldata() function and pg_controldata view added > > I don't think dumping out whatever pg_controldata happens to print as a > bunch of text fields is very sophisticated. We have functi

Re: [HACKERS] src/test/ssl broken on HEAD

Andrew Dunstan wrote: > > > On 09/02/2015 04:22 PM, Robert Haas wrote: > >For so long as this test suite is not run by 'make check-world' or by > >the buildfarm, it's likely to keep getting broken, and we're likely to > >keep not noticing. I realize that the decision to exclude this from > >'ma

Re: [HACKERS] src/test/ssl broken on HEAD

On Wed, Sep 2, 2015 at 4:50 PM, Andrew Dunstan wrote: > Tell me what's needed and I'll look at creating a buildfarm test module for > it. You run the tests via: make -C src/test/ssl check But nota bene security caveats: commit e39250c644ea7cd3904e4e24570db21a209cf97f Author: Heikki Linnakangas

Re: [HACKERS] psql tabcomplete - minor bugfix - tabcomplete for SET ROLE TO xxx

Hi 2015-09-02 15:23 GMT+02:00 Andres Freund : > Hi, > > On 2015-07-08 14:50:37 +0200, Pavel Stehule wrote: > > - static const char *const my_list[] = > > - {"DEFAULT", NULL}; > > + /* fallback for GUC settings */ > > > > -

Re: [HACKERS] src/test/ssl broken on HEAD

On 09/02/2015 04:22 PM, Robert Haas wrote: On Wed, Aug 26, 2015 at 3:37 AM, Michael Paquier wrote: On Wed, Aug 26, 2015 at 4:35 PM, Michael Paquier wrote: Only HEAD is impacted, and attached is a patch to fix the problem. Actually this version is better, I forgot to update a comment. For

Re: [HACKERS] ON CONFLICT DO UPDATE using EXCLUDED.column gives an error about mismatched types

On Wed, Sep 2, 2015 at 1:18 AM, Amit Langote wrote: > Did you get around to making a patch for this? I've worked on it inconsistently. I'll pick this up again soon. I may take the opportunity to talk this over with Andres in person when we meet at Postgres Open shortly. -- Peter Geoghegan --

Re: [HACKERS] src/test/ssl broken on HEAD

On Wed, Aug 26, 2015 at 3:37 AM, Michael Paquier wrote: > On Wed, Aug 26, 2015 at 4:35 PM, Michael Paquier > wrote: >> Only HEAD is impacted, and attached is a patch to fix the problem. > > Actually this version is better, I forgot to update a comment. For so long as this test suite is not run b

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/09/2015 15:53, Andres Freund wrote: > On 2015-07-18 12:17:39 +0200, Julien Rouhaud wrote: > > You also didn't touch /* * How many buffers PrefetchBuffer callers > should try to stay ahead of their * ReadBuffer calls by. This is > maintained by

Re: [HACKERS] Proposing COPY .. WITH PERMISSIVE

On 09/02/2015 10:10 PM, dinesh kumar wrote: > On Tue, Sep 1, 2015 at 10:58 PM, Stefan Kaltenbrunner > mailto:ste...@kaltenbrunner.cc>> wrote: > > On 07/25/2015 03:38 AM, dinesh kumar wrote: > > > > > > On Fri, Jul 24, 2015 at 10:22 AM, Robert Haas >

Re: [HACKERS] Proposing COPY .. WITH PERMISSIVE

On Tue, Sep 1, 2015 at 10:58 PM, Stefan Kaltenbrunner < ste...@kaltenbrunner.cc> wrote: > On 07/25/2015 03:38 AM, dinesh kumar wrote: > > > > > > On Fri, Jul 24, 2015 at 10:22 AM, Robert Haas > > wrote: > > > > On Thu, Jul 23, 2015 at 8:15 PM, dinesh kumar > >

Re: [HACKERS] Horizontal scalability/sharding

On 09/02/2015 08:27 PM, Robert Haas wrote: On Wed, Sep 2, 2015 at 1:59 PM, Merlin Moncure wrote: This strikes me as a bit of a conflict of interest with FDW which seems to want to hide the fact that it's foreign; the FDW implementation makes it's own optimization decisions which might make s

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On 9/2/15 2:17 PM, Alvaro Herrera wrote: Michael Paquier wrote: I haven't written yet a test case but I think that we could reproduce that simply by having a relation referenced in the exception block of a first function, calling a second function that itself raises an exception, causing the re

Re: [HACKERS] Allow replication roles to use file access functions

Michael Paquier wrote: > Hi all, > > As of now, file access functions in genfile.c can only be used by > superusers. This proposal is to relax those functions so as > replication users can use them as well. Here are the functions aimed > by this patch: > - pg_stat_file > - pg_read_binary_file > -

Re: [HACKERS] Pg_upgrade remote copy

Bruce Momjian wrote: > On Fri, Aug 28, 2015 at 10:34:38PM -0700, AI Rumman wrote: > > In pg_upgrade, how about adding a feature to copy data directory over > > network. > > That is, we can run pg_upgrade from our new host, where old host will be a > > remote machine. > I think it is much simpler

Re: [HACKERS] [PATCH] SQL function to report log message

On Mon, Aug 31, 2015 at 10:08 PM, Pavel Stehule wrote: > > > 2015-09-01 6:59 GMT+02:00 Pavel Stehule : > >> >> >> 2015-08-31 20:43 GMT+02:00 dinesh kumar : >> >>> Hi, >>> >>> On Sat, Aug 29, 2015 at 4:22 PM, Pavel Stehule >>> wrote: >>> Hi I am starting to work review of this patc

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 3:03 PM, Josh Berkus wrote: >> 4. Therefore, I think that we should instead use logical replication, >> which might be either synchronous or asynchronous. When you modify >> one copy of the data, that change will then be replicated to all other >> nodes. If you are OK with

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On Wed, Sep 2, 2015 at 7:12 AM, Andres Freund wrote: > On 2015-07-19 16:34:52 -0700, Peter Geoghegan wrote: > Hm. Is a compiler test actually test anything reliably here? Won't this > just throw a warning during compile time about an unknown function? I'll need to look into that. >> +/* >> + * P

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

Michael Paquier wrote: > I haven't written yet a test case but I think that we could reproduce > that simply by having a relation referenced in the exception block of > a first function, calling a second function that itself raises an > exception, causing the referencing error. Hm, so function 2

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, On 02/09/2015 18:06, Tomas Vondra wrote: > Hi > > On 09/02/2015 03:53 PM, Andres Freund wrote: >> >> Hi, >> >> On 2015-07-18 12:17:39 +0200, Julien Rouhaud wrote: >>> I didn't know that the thread must exists on -hackers to be >>> able to add a

Re: [HACKERS] Horizontal scalability/sharding

On 09/02/2015 11:41 AM, Robert Haas wrote: > On Wed, Sep 2, 2015 at 1:57 PM, Josh Berkus wrote: >> Even if it's only on paper, any new sharding design needs to address >> these questions: >> >> 1. How do we ensure no/minimal data is lost if we lose a node? >> 2. How do we replace a lost node (with

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Wed, Sep 2, 2015 at 4:52 AM, Shulgin, Oleksandr wrote: > On Tue, Sep 1, 2015 at 8:12 PM, Andres Freund wrote: >> >> On 2015-09-01 14:07:19 -0400, Robert Haas wrote: >> > But I think it's quite wrong to assume that the infrastructure for >> > this is available and usable everywhere, because in

Re: [HACKERS] FSM versus GIN pending list bloat

On Mon, Aug 10, 2015 at 1:16 PM, Jeff Janes wrote: > I have a simple test case that inserts an array of 101 md5 digests into each > row. With 10_000 of these rows inserted into an already indexed table, I > get 40MB for the table and 80MB for the index unpatched. With the patch, I > get 7.3 MB f

Re: [HACKERS] [PATCH] SQL function to report log message

On Tue, Sep 1, 2015 at 6:13 PM, Jim Nasby wrote: > My thought is that there's a fair amount of places where we do string > comparison for not a great reason. Perhaps a better example is data that > comes back from a trigger; AFTER/BEFORE, INSERT/UPDATE/..., which is more > expensive to setup the v

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On 2 Sep 2015 14:54, "Andres Freund" wrote: > > > > + /*-- > > + * The user-visible GUC parameter is the number of drives (spindles), > > + * which we need to translate to a number-of-pages-to-prefetch target. > > + * The target value is stashed in *extra and then assign

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

On Tue, Sep 1, 2015 at 6:43 PM, and...@anarazel.de wrote: > Why a new tranche for each of these? And it can't be correct that each > has the same base? I complained about the same-base problem before. Apparently, that got ignored. > I don't really like the tranche model as in the patch right no

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 1:57 PM, Josh Berkus wrote: > Even if it's only on paper, any new sharding design needs to address > these questions: > > 1. How do we ensure no/minimal data is lost if we lose a node? > 2. How do we replace a lost node (without taking the cluster down)? >2. a. how do we

Re: [HACKERS] pgbench stats per script & other stuff

On Wed, Sep 2, 2015 at 2:20 PM, Fabien COELHO wrote: >> I'm wondering if percentages instead of weights would be a better >> idea. That'd mean you'd be forced to be more careful when adding another >> script (having to adjust the percentages of other scripts) but arguably >> that's a good thing? >

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 1:59 PM, Merlin Moncure wrote: > On Tue, Sep 1, 2015 at 11:18 AM, Robert Haas wrote: >> It would be a bad idea to cling blindly to the FDW infrastructure if >> it's fundamentally inadequate to do what we want. On the other hand, >> it would also be a bad idea to set about

Re: [HACKERS] pgbench stats per script & other stuff

Hello Andres, Maybe add --builtin list to show them? Yep, easy enough. [...] +Shorthand for -b simple-update@1. +Shorthand for -b select-only@1. I'm a bit inclined to remove these options. Hm... This is really backward compatibility, and people may find reference to th

Re: Hooking at standard_join_search (Was: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual)

I wrote: > ... I imagine that we should allow FDWs to > store some data within RelOptInfo structs that represent foreign joins > belonging entirely to them, so that there'd be a handy place to keep that > data till later. Actually, if we do that (ie, provide a "void *fdw_state" field in join RelOp

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 11:18 AM, Robert Haas wrote: > It would be a bad idea to cling blindly to the FDW infrastructure if > it's fundamentally inadequate to do what we want. On the other hand, > it would also be a bad idea to set about recreating it without a > really good reason, and - just to

Re: [HACKERS] Horizontal scalability/sharding

On 09/01/2015 04:14 PM, Petr Jelinek wrote: > On 2015-09-02 00:09, Josh Berkus wrote: >> On 09/01/2015 02:29 PM, Tomas Vondra wrote: >>> So while you may be right in single-DC deployments, with multi-DC >>> deployments the situation is quite different - not only that the network >>> bandwidth is no

Re: Hooking at standard_join_search (Was: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual)

Robert Haas writes: > On Wed, Sep 2, 2015 at 10:30 AM, Tom Lane wrote: >> But if you have in mind that typical FDWs would actually create join paths >> at that point, consider that >> >> 1. The FDW would have to find all the combinations of its supplied >> relations (unless you are only intendin

Re: [HACKERS] Improving test coverage of extensions with pg_dump

Andres Freund wrote: > > As I recall, Andrew Dunstan has a module that > > tests cross-version pg_upgrade and one thing he does is dump both and > > compare; the problem is that there are differences, so he keeps a count > > of how many lines he expect to differ between any two releases. > > I'm

Re: [HACKERS] Improving test coverage of extensions with pg_dump

On 2015-09-02 14:30:33 -0300, Alvaro Herrera wrote: > I was thinking in having this be renamed src/test/modules/extensions/ > and then the extension contained here would be renamed ext001_fk_tables > or something like that; later we could ext002_something for testing some > other angle of extension

Re: [HACKERS] Improving test coverage of extensions with pg_dump

Andres Freund wrote: > Isn't a full test with a separate initdb, create extension etc. a really > heavyhanded way to test this? I mean that's a test where the setup takes > up to 10s, whereas the actual runtime is in the millisecond range? I spent some time looking over this patch yesterday, and

Re: [HACKERS] pgbench stats per script & other stuff

On 2015-07-30 18:03:56 +0200, Fabien COELHO wrote: > > >v6 is just a rebase after a bug fix by Andres Freund. > > > >Also a small question: The patch currently displays pgbench scripts > >starting numbering at 0. Probably a little too geek... should start at 1? > > v7 is a rebase after another sm

  1   2   >