[HACKERS] CREATE STATISTICS statistic_type documentation

2017-05-25 Thread Jeff Janes
The docs for CREATE STATISTICS does not say what happens if the statistic_type clause is omitted. It should probably say that the default action is to create both ndistinct and dependencies. Cheers, Jeff

Re: [HACKERS] CREATE STATISTICS statistic_type documentation

2017-05-25 Thread Jeff Janes
On Thu, May 25, 2017 at 9:28 AM, Tom Lane wrote: > Jeff Janes writes: > > The docs for CREATE STATISTICS does not say what happens if the > > statistic_type clause is omitted. It should probably say that the > default > > action is to create both ndistinct and depende

Re: [HACKERS] logical replication - still unstable after all these months

2017-05-26 Thread Jeff Janes
subscriber ends up with about twice as many rows. Cheers, Jeff

Re: [HACKERS] logical replication - still unstable after all these months

2017-05-26 Thread Jeff Janes
ffer in the *balance column or something else? Are they transactionally consistent? I have not been able to replicate the problem. Cheers, Jeff

[HACKERS] logical replication busy-waiting on a lock

2017-05-26 Thread Jeff Janes
nd so on out to "after 9616.814", when it finally acquires the lock. The other process, 47457, is doing the initial COPY of another table as part of the same publisher/subscriber set. Cheers, Jeff

[HACKERS] simplehash.h typo

2017-05-27 Thread Jeff Janes
/* round up size to the next power of 2, that's the bucketing works */ That should probably be "that's the **way** bucketing works". Or maybe it is an idiom I don't grok. Cheers, Jeff

[HACKERS] execGrouping.c limit on work_mem

2017-05-27 Thread Jeff Janes
. Now we will just do a bunch of hash-table splitting in the process. That is only going to add to the pain. Also: * false if it existed already. ->additional_data in the new entry has The field is just ->additional, not ->additional_data Cheers, Jeff

Re: [HACKERS] logical replication - still unstable after all these months

2017-05-28 Thread Jeff Janes
t there is some bug being tickled here. > Unfortunately the test framework blasts away the failed tables and > subscription and continues on...I'm going to amend it to stop on failure so > I can have a closer look at what happened. > What would you want to look at? Would saving the WAL from the master be helpful? Cheers, Jeff

Re: [HACKERS] logical replication busy-waiting on a lock

2017-05-29 Thread Jeff Janes
> g_0xmuc%3DUpBrZ-_MUxh-Q%40mail.gmail.com Cheers, Jeff

Re: [HACKERS] psql: Activate pager only for height, not width

2017-05-29 Thread Jeff Janes
kick in if greater than some settable number of the lines are too wide, or if the wrapped lines would push the height above the height limit. If just one line is too wide, I'd rather just deal with them being wrapped. (You can configure the pager not to redraw the screen when exited, but I want it to redraw the screen after looking at 10,000 rows, just not after looking ten rows, one of which was 170 characters wide) Cheers, Jeff

Re: [HACKERS] Hash Functions

2017-06-01 Thread Jeff Davis
fine with either option. > 2. Add an option like --dump-partition-data-with-parent. I'm not sure > who originally proposed this, but it seems that everybody likes it. > What we disagree about is the degree to which it's sufficient. Jeff > Davis thinks it doesn't go

Re: [HACKERS] Hash Functions

2017-06-01 Thread Jeff Davis
o the latter. Once we support more pushdowns to partitions, the only question is: what are your join keys and what are your grouping keys? Text is absolutely a normal join key or group key. Consider joins on a user ID or grouping by a model number. Regards, Jeff Davis -- Sent via pgsq

Re: [HACKERS] Range Merge Join v1

2017-06-02 Thread Jeff Davis
On Tue, May 30, 2017 at 11:44 PM, Andrew Borodin wrote: > Hi, Jeff! Hi! > Sorry for being late. Actually, I had several unsuccessful attempts to > find something wrong with the patch. > Here's my review. > > in pathkey.c > > ecs = (EquivalenceClass **) palloc(nClau

Re: [HACKERS] logical replication - still unstable after all these months

2017-06-03 Thread Jeff Janes
> to the issue you've seen though. > This conflicts again with Peter E's recent commit 3c9bc2157a4f465b3c070d1250597568d2dc285f Thanks, Jeff

[HACKERS] postgres_fdw cost estimation defaults and documentation

2017-06-05 Thread Jeff Janes
nning purposes. Perhaps adding something like "Also, local statistics do not contain information on the available indexes on the remote side, while use_remote_estimate does take these into account"? Cheers, Jeff

Re: [HACKERS] List of hostaddrs not supported

2017-06-09 Thread Jeff Janes
me compiler warnings: fe-connect.c: In function 'connectDBStart': fe-connect.c:1625: warning: 'ret' may be used uninitialized in this function gcc version 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) Cheers, Jeff

[HACKERS] partial aggregation with internal state type

2017-06-09 Thread Jeff Janes
n the aggregate memory context." Since combinefunc with an internal type is only useful when serialfunc and deserialfunc are also defined, why can't the built-in machinery just do the right thing when faced with a strict combinefunc? Cheers, Jeff

Re: [HACKERS] partial aggregation with internal state type

2017-06-09 Thread Jeff Janes
On Fri, Jun 9, 2017 at 9:06 AM, Tom Lane wrote: > Jeff Janes writes: > > The docs for creating aggregates for 9.6 and beyond say: > > "For aggregate functions whose state_data_type is internal, the > combinefunc > > must not be strict. In this case the comb

Re: [HACKERS] List of hostaddrs not supported

2017-06-09 Thread Jeff Janes
On Fri, Jun 9, 2017 at 11:52 AM, Heikki Linnakangas wrote: > On 06/09/2017 05:47 PM, Jeff Janes wrote: > >> Your commit to fix this part, 76b11e8a43eca4612d, is giving me compiler >> warnings: >> >> fe-connect.c: In function 'connectDBStart': >>

[HACKERS] logical replication NOTICE "synchronized table states"

2017-06-09 Thread Jeff Janes
ing me the data had finished copying over (i.e. the initial state of the table data had been synced). But at the time this notice is issued, the copy has not even started. In fact, if I create a subscription for a non-existent publication, I still get this message. Cheers, Jeff

[HACKERS] logical replication: \dRp+ and "for all tables"

2017-06-09 Thread Jeff Janes
ppens to include all the table which currently exist. That seems unfortunate. Should the "for all tables" be included as another column in \dRp and \dRp+, or at least as a footnote tag in \dRp+ ? Cheers, Jeff

Re: [HACKERS] logical replication: \dRp+ and "for all tables"

2017-06-09 Thread Jeff Janes
On Fri, Jun 9, 2017 at 10:20 PM, Masahiko Sawada wrote: > On Sat, Jun 10, 2017 at 7:29 AM, Jeff Janes wrote: > > If I create a publication "for all tables", \dRp+ doesn't indicate it is > for > > all tables, it just gives a list of the tables. > >

Re: [HACKERS] Why restore_command is called for existing files in pg_xlog?

2017-06-12 Thread Jeff Janes
systems. You have do a manual inspection of the differences. To do that, you would start by starting up (a copy of) the crashed master, using normal crash recovery, not PITR. > > Is this a correct list for such questions, or would it be more appropriate > to ask elsewhere (i.e. pgsql-bugs?) > Probably more appropriate for pgsql-general or pgsql-admin. Cheers, Jeff

Re: [HACKERS] logical replication: \dRp+ and "for all tables"

2017-06-12 Thread Jeff Janes
On Sat, Jun 10, 2017 at 7:42 AM, Tom Lane wrote: > Jeff Janes writes: > > On Fri, Jun 9, 2017 at 10:20 PM, Masahiko Sawada > > wrote: > >> On Sat, Jun 10, 2017 at 7:29 AM, Jeff Janes > wrote: > >>> That seems unfortunate. Should the "for all ta

[HACKERS] subscription worker signalling wal writer too much

2017-06-14 Thread Jeff Janes
l backends can share it, at the expense of additional work needed to be done under a spin lock. Other ideas? Thanks, Jeff

Re: [HACKERS] subscription worker signalling wal writer too much

2017-06-14 Thread Jeff Janes
On Wed, Jun 14, 2017 at 11:55 AM, Jeff Janes wrote: > If I publish a pgbench workload and subscribe to it, the subscription > worker is signalling the wal writer thousands of times a second, once for > every async commit. This has a noticeable performance cost. > I've used a

Re: [HACKERS] subscription worker signalling wal writer too much

2017-06-14 Thread Jeff Janes
On Wed, Jun 14, 2017 at 3:20 PM, Andres Freund wrote: > On 2017-06-14 15:08:49 -0700, Jeff Janes wrote: > > On Wed, Jun 14, 2017 at 11:55 AM, Jeff Janes > wrote: > > > > > If I publish a pgbench workload and subscribe to it, the subscription > > > worker is

Re: [HACKERS] subscription worker signalling wal writer too much

2017-06-15 Thread Jeff Janes
On Wed, Jun 14, 2017 at 4:29 PM, Andres Freund wrote: > On 2017-06-14 16:24:27 -0700, Jeff Janes wrote: > > On Wed, Jun 14, 2017 at 3:20 PM, Andres Freund > wrote: > > > > > On 2017-06-14 15:08:49 -0700, Jeff Janes wrote: > > > > On Wed, Jun 14, 20

Re: [HACKERS] subscription worker signalling wal writer too much

2017-08-26 Thread Jeff Janes
On Mon, Jul 3, 2017 at 8:26 PM, Jeff Janes wrote: > On Sat, Jun 24, 2017 at 5:09 PM, Andres Freund wrote: > >> On 2017-06-15 15:06:43 -0700, Jeff Janes wrote: >> >> >> > >> > Wouldn't it >> > better, and much simpler, just to have revert

[HACKERS] pgbench: faster version of tpcb-like transaction

2017-08-26 Thread Jeff Janes
er-defined transaction with -f, you have to go out of your way to create the function (no "-i" support) and to make sure :scale gets set correctly during runs (as it won't be automatically read from pgbench_branches table, you have manually give -D). Cheers, Jeff pgbench_functio

Re: [HACKERS] pgbench: faster version of tpcb-like transaction

2017-08-26 Thread Jeff Janes
On Sat, Aug 26, 2017 at 4:28 PM, Peter Geoghegan wrote: > On Sat, Aug 26, 2017 at 3:53 PM, Jeff Janes wrote: > > I get nearly a 3 fold speed up using the new transaction, from 9184 to > 26383 > > TPS, on 8 CPU machine using scale 50 and: > > > > PGOPTIONS="

Re: [HACKERS] Range Merge Join v1

2017-08-31 Thread Jeff Davis
On Fri, Aug 25, 2017 at 10:19 AM, Alexander Kuzmenkov wrote: > Hi Jeff, Hi, Thank you for the review and suggestions! > * At the moment, "mergejoinable clause" and "equality clause" mean the same > thing to the planner, and those clauses are used both to create

[HACKERS] pg_basebackup throttling doesn't throttle as promised

2017-09-01 Thread Jeff Janes
r both? Cheers, Jeff

Re: [HACKERS] pg_basebackup throttling doesn't throttle as promised

2017-09-01 Thread Jeff Janes
On Fri, Sep 1, 2017 at 1:32 PM, Jeff Janes wrote: > The "-r" option to pg_basebackup is supposed to throttle the rate of the > backup. But it only works properly if the server is mostly idle. > > Every non-trivial call to XLogFlush or XLogBackgroundFlush will wake up >

Re: [HACKERS] pgbench - minor fix for meta command only scripts

2017-09-04 Thread Jeff Janes
m as "break". Pass-throughs are error-prone, and make it more > difficult to read, IMHO. The compiler will optimize it into a pass-through > anyway, if possible and worthwhile, so there should be no performance > difference. Since this commit (12788ae49e1933f463bc5), if I use the --rate to throttle the transaction rate, it does get throttled to about the indicated speed, but the pg_bench consumes the entire CPU. At the block of code starting if (min_usec > 0 && maxsock != -1) If maxsock == -1, then there is no sleep happening. Cheers, Jeff

Re: [HACKERS] pgbench - minor fix for meta command only scripts

2017-09-04 Thread Jeff Janes
On Mon, Sep 4, 2017 at 1:56 PM, Fabien COELHO wrote: > > Hello Jeff, > > I have fixed a bug introduced in the patch by changing && by || in the >>>> (min_sec > 0 && maxsock != -1) condition which was inducing errors with >>>>

[HACKERS] pg_basebackup behavior on non-existent slot

2017-09-04 Thread Jeff Janes
oobar -Xs pg_basebackup: could not send replication command "START_REPLICATION": ERROR: replication slot "foobar" does not exist 22384213/22384213 kB (100%), 1/1 tablespace pg_basebackup: child process exited with error 1 pg_basebackup: removing data directory "data_replica" Cheers, Jeff

Re: [HACKERS] Fix performance of generic atomics

2017-09-05 Thread Jeff Janes
chronous_commit 'on' and 'off' using > both logged and unlogged tables. Also ran an internal benchmark which > didn't show anything either. > What scale factor and client count? How many cores per socket? It looks like Sokolov was just starting to see gains at 200 clients on 72 cores, using -N transaction. Cheers, Jeff

Re: [HACKERS] Create replication slot in pg_basebackup if requested and not yet present

2017-09-08 Thread Jeff Janes
change that. > +1. I'd rather just get an error and re-run without the --create switch. That way you are forced to think about what you are doing. Is there a race condition here? The slot is created after the checkpoint is completed. But you have to start streaming from the LSN where the checkpoint started, so shouldn't the slot be created before the checkpoint is started? Cheers, Jeff

Re: [HACKERS] Automatic testing of patches in commit fest

2017-09-10 Thread Jeff Janes
ailing" icon, it takes me to a generic page, not one describing how that specific build is failing. Cheers, Jeff

Re: [HACKERS] pgbench - minor fix for meta command only scripts

2017-09-11 Thread Jeff Janes
On Mon, Sep 11, 2017 at 1:49 AM, Fabien COELHO wrote: > > Hello Jeff, > > Ok, the problem was a little bit more trivial than I thought. > > The issue is that under a low rate there may be no transaction in > progress, however the wait procedure was relying on select'

Re: [HACKERS] pg_basebackup behavior on non-existent slot

2017-09-12 Thread Jeff Janes
On Wed, Sep 6, 2017 at 2:50 AM, Alvaro Herrera wrote: > Magnus Hagander wrote: > > On Mon, Sep 4, 2017 at 3:21 PM, Jeff Janes wrote: > > > > Should the parent process of pg_basebackup be made to respond to > SIGCHLD? > > > Or call waitpid(bgchild, &st

[HACKERS] uninterruptible state in 10beta4

2017-09-13 Thread Jeff Janes
in BackendRun (argc=, argv=) at postmaster.c:4357 #20 BackendStartup (argc=, argv=) at postmaster.c:4029 #21 ServerLoop (argc=, argv=) at postmaster.c:1753 #22 PostmasterMain (argc=, argv=) at postmaster.c:1361 #23 0x00631410 in main (argc=1, argv=0x141c2f0) at main.c:228 Cheers, Jeff

Re: [HACKERS] uninterruptible state in 10beta4

2017-09-13 Thread Jeff Janes
On Wed, Sep 13, 2017 at 2:41 PM, Andres Freund wrote: > Hi, > > On 2017-09-13 14:28:34 -0700, Jeff Janes wrote: > > In 10beta4 and 11dev, If I run the below it enters an uninterruptible > > state. After the insert starts, I give 15 or seconds or so until the > > memo

Re: [HACKERS] postgres_fdw super user checks

2017-09-14 Thread Jeff Janes
On Tue, Sep 12, 2017 at 1:13 AM, Andreas Karlsson wrote: > On 07/27/2017 09:45 PM, Jeff Janes wrote:> Here is an updated patch. This > version allows you use the password-less > >> connection if you either are the super-user directly (which is the >> existing committed

Re: [HACKERS] GnuTLS support

2017-09-15 Thread Jeff Janes
to openssl server and the reverse (and gnutls to gnutls). Cheers, Jeff

Re: [HACKERS] CLUSTER command progress monitor

2017-09-15 Thread Jeff Janes
l | bigint | | | > heap_tuples_scanned | bigint | | | > I think it should be cluster_index_relid, not scan_index_relid. If the scan_method is seq, then the index isn't being scanned. Cheers, Jeff

Re: [HACKERS] Range Merge Join v1

2017-09-17 Thread Jeff Davis
On Thu, Aug 31, 2017 at 1:52 AM, Jeff Davis wrote: > Updated patch attached. Changelog: > > * Rebased > * Changed MJCompare to return an enum as suggested, but it has 4 > possible values rather than 3. > * Added support for joining on contains or contained by (@> or <@) a

Re: [HACKERS] GnuTLS support

2017-09-18 Thread Jeff Janes
On Sun, Sep 17, 2017 at 2:17 PM, Andreas Karlsson wrote: > On 09/15/2017 06:55 PM, Jeff Janes wrote: > >> I can't build against gnutls-2.12.23-21.el6.x86_64 from CentOS 6.9 >> > > Thanks for testing my patch. I have fixed both these issues plus some of > the oth

Re: [HACKERS] Create replication slot in pg_basebackup if requested and not yet present

2017-09-19 Thread Jeff Janes
the WAL > > streamer connection, but this would keep any fragility around for > > (likely more frequently used) temporary replication slots. It would make > > the patch much smaller though if I revert touching temporary slots at > > all. > > That's what I was thinki

Re: [HACKERS] why not parallel seq scan for slow functions

2017-09-19 Thread Jeff Janes
t=0.00..625383.00 rows=2500 > width=8) > (3 rows) > that should be fixed by turning costs on the explain, as is the tradition. See attached. Cheers, Jeff parallel_paths_include_tlist_cost_v4.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@p

Re: [HACKERS] SCRAM in the PG 10 release notes

2017-09-19 Thread Jeff Janes
)? > > I guess I'm late to the party, but I don't see why this is needed at all. We encourage people to use any and all new features which are appropriate to them--that is why we implement new features. Why does this feature need a special invitation? Cheers, Jeff

Re: [HACKERS] SCRAM in the PG 10 release notes

2017-09-19 Thread Jeff Janes
On Tue, Sep 19, 2017 at 4:29 PM, Michael Paquier wrote: > On Wed, Sep 20, 2017 at 6:55 AM, Jeff Janes wrote: > > On Tue, Sep 19, 2017 at 1:32 PM, Heikki Linnakangas > wrote: > >> I'm not sure what exactly to do here. Where should we stick that notice? > >> W

Re: [HACKERS] SCRAM in the PG 10 release notes

2017-09-20 Thread Jeff Janes
uch less likely to be kept up to date. The problem with it currently is that it implies anything using libpq supports scram, even though a libpq which supports scram has not even been released yet. Cheers, Jeff

Re: [HACKERS] why not parallel seq scan for slow functions

2017-09-20 Thread Jeff Janes
On Tue, Sep 19, 2017 at 9:15 PM, Amit Kapila wrote: > On Wed, Sep 20, 2017 at 3:05 AM, Jeff Janes wrote: > > On Tue, Sep 19, 2017 at 1:17 PM, Thomas Munro > > wrote: > >> > >> On Thu, Sep 14, 2017 at 3:19 PM, Amit Kapila > >> wrote: > >> >

Re: [HACKERS] SCRAM in the PG 10 release notes

2017-09-21 Thread Jeff Janes
On Thu, Sep 21, 2017 at 7:42 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 9/19/17 20:45, Peter Eisentraut wrote: > > On 9/19/17 17:55, Jeff Janes wrote: > >> I guess I'm late to the party, but I don't see why this is needed at > >&g

[HACKERS] 10RC1 crash testing MultiXact oddity

2017-09-22 Thread Jeff Janes
that this should only happen on a server that has been upgraded from 9.3 or 9.4, which this server has not been. Is the presence of this log message something that needs to be investigated further? Thanks, Jeff 0002-pg_burn_multixact-utility_v10.patch Description: Binary data count.pl

Re: [HACKERS] 10RC1 crash testing MultiXact oddity

2017-09-22 Thread Jeff Janes
On Fri, Sep 22, 2017 at 8:47 AM, Alvaro Herrera wrote: > Jeff Janes wrote: > > I am running some crash recovery testing against 10rc1 by injecting torn > > page writes, using a test case which generates a lot of multixact, some > > naturally by doing a lot fk updates, but

[HACKERS] v10 pg_ctl compatibility

2017-09-26 Thread Jeff Janes
ather confusing (waiting for a long time, and then reporting failure, even though it started successfully). Cheers, Jeff

Re: [HACKERS] v10 pg_ctl compatibility

2017-09-26 Thread Jeff Janes
On Tue, Sep 26, 2017 at 12:29 PM, Andres Freund wrote: > Hi, > > On 2017-09-26 11:59:42 -0700, Jeff Janes wrote: > > Should the release notes have a compatibility entry about pg_ctl restart, > > being used against a running pre-10 server, no longer being able to > de

Re: [HACKERS] v10 pg_ctl compatibility

2017-09-26 Thread Jeff Janes
On Tue, Sep 26, 2017 at 1:10 PM, Tom Lane wrote: > Jeff Janes writes: > > To add insult to injury, when v10 pg_ctl does restart a pre-10 server and > > it sits there for a long time waiting for it to start up even though it > has > > already started up, if I hit

Re: [HACKERS] v10 pg_ctl compatibility

2017-09-26 Thread Jeff Janes
On Tue, Sep 26, 2017 at 3:54 PM, Tom Lane wrote: > Jeff Janes writes: > > On Tue, Sep 26, 2017 at 1:10 PM, Tom Lane wrote: > >> Really? The server should have detached itself from your terminal > >> group long before that. What platform is this? > > > Cen

Re: [HACKERS] v10 pg_ctl compatibility

2017-09-26 Thread Jeff Janes
On Tue, Sep 26, 2017 at 4:31 PM, Tom Lane wrote: > Jeff Janes writes: > > I was not using -l. Instead I set logging_collector=on in > postgresql.conf, > > but I suppose that that is not sufficent? > > No, because initial stderr is still connected to whatever. > >

Re: [HACKERS] pgbench - minor fix for meta command only scripts

2017-09-29 Thread Jeff Janes
On Mon, Sep 11, 2017 at 6:27 PM, Fabien COELHO wrote: > > Hello Jeff, > > Shouldn't we use pg_usleep to ensure portability? it is defined for >> front-end code. But it returns void, so the error check will have to be >> changed. >> > > Attached v3 wi

Re: [HACKERS] 10RC1 crash testing MultiXact oddity

2017-09-30 Thread Jeff Janes
On Fri, Sep 22, 2017 at 1:19 PM, Robert Haas wrote: > On Fri, Sep 22, 2017 at 3:39 PM, Jeff Janes wrote: > > It turns out it is not new in pg10. I spotted in the log file only by > > accident while looking for something else. Now that I am looking for > it, I > >

Re: [HACKERS] LWLock deadlock and gdb advice

2015-07-19 Thread Jeff Janes
ys waits > if the flag is not set, i.e. it will not return regardless of the > variable's value, if the current lock-holder has not updated it yet. > > This passes make check, but I haven't done any testing beyond that. Does > this look sane to you? After applying this patch to commit fdf28853ae6a397497b79f, it has survived testing long enough to convince that this fixes the problem. Cheers, Jeff

Re: [HACKERS] LWLock deadlock and gdb advice

2015-07-19 Thread Jeff Janes
On Thu, Jul 16, 2015 at 12:03 AM, Jeff Janes wrote: > On Wed, Jul 15, 2015 at 8:44 AM, Heikki Linnakangas > wrote: > >> >> Both. Here's the patch. >> >> Previously, LWLockAcquireWithVar set the variable associated with the >> lock atomically with

Re: [HACKERS] brin index vacuum versus transaction snapshots

2015-07-21 Thread Jeff Janes
On Tue, Jul 21, 2015 at 3:10 PM, Kevin Grittner wrote: > Kevin Grittner wrote: > > > If you run `make installcheck` against a cluster with > > default_transaction_isolation = 'repeatable read' you get one > > failure: > > > + ERROR: brin_summarize_new_values() cannot run in a transaction that >

Re: [HACKERS] optimizing vacuum truncation scans

2015-07-22 Thread Jeff Janes
On Wed, Jul 22, 2015 at 6:59 AM, Robert Haas wrote: > On Mon, Jun 29, 2015 at 1:54 AM, Jeff Janes wrote: > > Attached is a patch that implements the vm scan for truncation. It > > introduces a variable to hold the last blkno which was skipped during the > > forward porti

Re: [HACKERS] All-zero page in GIN index causes assertion failure

2015-07-23 Thread Jeff Janes
as a page > that's marked as deleted must also be empty, and wherever we check for the > deleted-flag, we also check for PageIsEmpty())) This patch, in conjunction with the LWLock deadlock patch, fixes all the issues I was having with GIN indexes in 9.5. I haven't tried SP-GiST. Cheers, Jeff

Re: [HACKERS] "make check" changes have caused buildfarm deterioration.

2015-07-24 Thread Jeff Janes
irect it > only to abs_top_builddir/log/ but tmp_install is not removed after a > run of a regression make target. > If I run 'make check' on an unbuilt tree, any compiler warnings emitted during the build phase now get directed into the install log. Was that intentional or a side effect? Cheers, Jeff

Re: [HACKERS] LWLock deadlock and gdb advice

2015-07-28 Thread Jeff Janes
On Tue, Jul 28, 2015 at 7:06 AM, Andres Freund wrote: > Hi, > > On 2015-07-19 11:49:14 -0700, Jeff Janes wrote: > > After applying this patch to commit fdf28853ae6a397497b79f, it has > survived > > testing long enough to convince that this fixes the problem. > >

Re: [HACKERS] LWLock deadlock and gdb advice

2015-07-29 Thread Jeff Janes
On Tue, Jul 28, 2015 at 9:06 AM, Jeff Janes wrote: > On Tue, Jul 28, 2015 at 7:06 AM, Andres Freund wrote: > >> Hi, >> >> On 2015-07-19 11:49:14 -0700, Jeff Janes wrote: >> > After applying this patch to commit fdf28853ae6a397497b79f, it has >> survived &g

Re: [HACKERS] LWLock deadlock and gdb advice

2015-07-29 Thread Jeff Janes
On Wed, Jul 29, 2015 at 9:26 AM, Andres Freund wrote: > On 2015-07-29 09:23:32 -0700, Jeff Janes wrote: > > On Tue, Jul 28, 2015 at 9:06 AM, Jeff Janes > wrote: > > I've reproduced it again against commit b2ed8edeecd715c8a23ae462. > > > > It took 5 hours o

Re: [HACKERS] LWLock deadlock and gdb advice

2015-07-30 Thread Jeff Janes
r queueing. Want to write the patch, or should I? > > > > I'll try. Shouldn't be too hard. > > What do you think about something roughly like the attached? > I've not evaluated the code, but applying it does solve the problem I was seeing. Cheers, Jeff

Re: [HACKERS] LWLock deadlock and gdb advice

2015-07-31 Thread Jeff Janes
On Thu, Jul 30, 2015 at 8:22 PM, Andres Freund wrote: > On 2015-07-30 09:03:01 -0700, Jeff Janes wrote: > > On Wed, Jul 29, 2015 at 6:10 AM, Andres Freund > wrote: > > > What do you think about something roughly like the attached? > > > > > > > I'

Re: [HACKERS] optimizing vacuum truncation scans

2015-08-03 Thread Jeff Janes
On Mon, Jul 27, 2015 at 1:40 PM, Simon Riggs wrote: > On 22 July 2015 at 17:11, Jeff Janes wrote: > >> On Wed, Jul 22, 2015 at 6:59 AM, Robert Haas >> wrote: >> >>> On Mon, Jun 29, 2015 at 1:54 AM, Jeff Janes >>> wrote: >>> > Attached is a

[HACKERS] FSM versus GIN pending list bloat

2015-08-03 Thread Jeff Janes
ng up one level and then not writing to it, assuming the pending list pages remain well clustered. Or would a completely different approach be better, like managing the vacated pending list pages directly in the index without going to the fsm? Cheers, Jeff gin_fast_freespace.patch Description: B

Re: [HACKERS] Re: Using quicksort and a merge step to significantly improve on tuplesort's single run "external sort"

2015-08-03 Thread Jeff Janes
ay we currently do it has an NlogN worst case, while the other way is linear even in the worst case. Cheers, Jeff

Re: [HACKERS] FSM versus GIN pending list bloat

2015-08-04 Thread Jeff Janes
On Tue, Aug 4, 2015 at 1:39 AM, Simon Riggs wrote: > On 4 August 2015 at 06:03, Jeff Janes wrote: > > >> The attached proof of concept patch greatly improves the bloat for both >> the insert and the update cases. You need to turn on both features: adding >> the page

Re: [HACKERS] FSM versus GIN pending list bloat

2015-08-04 Thread Jeff Janes
On Tue, Aug 4, 2015 at 6:35 AM, Simon Riggs wrote: > On 4 August 2015 at 09:39, Simon Riggs wrote: > >> On 4 August 2015 at 06:03, Jeff Janes wrote: >> >> >>> The attached proof of concept patch greatly improves the bloat for both >>> the insert and t

Re: [HACKERS] LWLock deadlock and gdb advice

2015-08-05 Thread Jeff Janes
out in other tests. I don't know why the gin test was efficient at provoking the problem while none of the other ones (btree-upsert, gist, brin, btree-foreign key, btree-prepare transaction) I've played around with. Perhaps it is just due to the amount of WAL which gin generates. Cheers, Jeff

Fwd: [HACKERS] 9.5 release notes

2015-08-06 Thread Jeff Janes
ic" refer specifically to the type "numeric", or to all fields which are numeric in nature? If the former (which I think it is, as other numeric types are pass-by-value to start with), it should probably get some sgml markup. (copy to list, accidentally sent just to Bruce initially) Thanks, Jeff

Re: [HACKERS] GIN pageinspect functions

2015-08-10 Thread Jeff Janes
page() function, and the other functions that > are designed to work with that, was added. gin_leafpage_items() name seems > fine to me. When I call gin_leafpage_items on a {leaf} page, I get the ERROR: ERROR: input page is not a compressed GIN data leaf page DETAIL: Flags 0002, expected 00

Re: PENDING_LIST_CLEANUP_SIZE - maximum size of GIN pending list Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2015-08-10 Thread Jeff Janes
er. It would be nice to try the fastupdate=off alternative (i.e. the same as pending_list_cleanup_size=0) without having to change the index itself and change the syntax used in the testing. Cheers, Jeff

Re: [HACKERS] FSM versus GIN pending list bloat

2015-08-10 Thread Jeff Janes
On Tue, Aug 4, 2015 at 12:38 PM, Jeff Janes wrote: > On Tue, Aug 4, 2015 at 1:39 AM, Simon Riggs wrote: > >> On 4 August 2015 at 06:03, Jeff Janes wrote: >> >> >>> The attached proof of concept patch greatly improves the bloat for both >>> the insert

[HACKERS] statistics for array types

2015-08-11 Thread Jeff Janes
--- Seq Scan on foobar (cost=0.00..2387.00 rows=3 width=61) (actual time=78.604..78.604 rows=0 loops=1) Filter: (foo @> '{567}'::integer[]) Rows Removed by Filter: 10 Planning time: 0.199 ms Execution time: 78.665 ms Cheers, Jeff array_type_analyze_

[HACKERS] GIN pending clean up is not interruptable

2015-08-11 Thread Jeff Janes
to call CHECK_FOR_INTERRUPTS(). But I think we could instead just call vacuum_delay_point unconditionally. It calls CHECK_FOR_INTERRUPTS(), and if not in a throttled vacuum it does nothing else. (That is how ANALYZE handles it.) This issue is in all branches. Cheers, Jeff

Re: [HACKERS] GIN pending clean up is not interruptable

2015-08-12 Thread Jeff Janes
On Tue, Aug 11, 2015 at 5:27 PM, Tom Lane wrote: > Andres Freund writes: > > On 2015-08-11 15:07:15 -0700, Jeff Janes wrote: > >> The attached patch adds an else branch to call CHECK_FOR_INTERRUPTS(). > >> > >> But I think we could instead just call vacuum_de

[HACKERS] GIN pending list clean up exposure to SQL

2015-08-12 Thread Jeff Janes
xisting code which I used as an exemplar for writing this code. But where does this belong? Core? Its own separate extension? Cheers, Jeff gin_clean_pending_user_v001.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to you

[HACKERS] Potential GIN vacuum bug

2015-08-15 Thread Jeff Janes
, without waiting for that someone else to finish the job. Isn't this a problem? Cheers, Jeff

Re: [HACKERS] Potential GIN vacuum bug

2015-08-17 Thread Jeff Janes
On Mon, Aug 17, 2015 at 6:23 AM, Jeff Janes wrote: > > On Aug 16, 2015 11:49 PM, "Heikki Linnakangas" wrote: > > > > On 08/16/2015 12:58 AM, Jeff Janes wrote: > >> > >> When ginbulkdelete gets called for the first time in a VACUUM(i.e. > stats

Re: [HACKERS] Potential GIN vacuum bug

2015-08-17 Thread Jeff Janes
On Mon, Aug 17, 2015 at 3:02 PM, Alvaro Herrera wrote: > Jeff Janes wrote: > > > The attached patch takes a ShareUpdateExclusiveLock lock on the index in > > order to clean the pending list. > > Does it take a lock on the table also? Because if not ... > There must b

[HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-18 Thread Jeff Janes
roblem in this area, please try it out or describe it. There are other places in tqual.c which could probably use similar re-ordering tricks, but this is the one for which I have a reproducible test case, Cheers Jeff SatisfiesMVCC_reorder_v001.patch Description: Binary data -- Sent via pgs

Re: [HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-19 Thread Jeff Janes
h or without frequent crashes) without signs of missed or duplicated rows. So, it looks good to me. Cheers, Jeff

Re: [HACKERS] Mention column name in error messages

2015-08-19 Thread Jeff Janes
n gives two new lines once they are emitted. This is contrary to how other HINTs are formatted. Other HINTs are complete sentences (start with a capital letter, end with a period). But I think these belong as CONTEXT or as DETAIL, not as HINT. The messages are giving me details about where (which column) the error occured, they aren't giving suggestions to me about what to do about it. Cheers, Jeff

Re: [HACKERS] Patch for ginCombineData

2015-08-21 Thread Jeff Janes
rk_mem and will flush it all. I think most modern OSes will not have a problems caused by allocated but untouched memory. This patch doesn't introduce that problem, it just allows it to operate at a higher absolute size. So I'm marking this as ready for committer. Cheers, Jeff

Re: [HACKERS] Potential GIN vacuum bug

2015-08-22 Thread Jeff Janes
On Tue, Aug 18, 2015 at 8:59 AM, Robert Haas wrote: > On Mon, Aug 17, 2015 at 5:41 PM, Jeff Janes wrote: > > User backends attempt to take the lock conditionally, because otherwise > they > > would cause an autovacuum already holding the lock to cancel itself, > whic

Re: [HACKERS] Test code is worth the space

2015-08-22 Thread Jeff Janes
on? > +1 on that. It would be helpful to see successful examples. Especially ones written in C. I can't really figure out what success looks like just from reading the descriptions. Cheers, Jeff

Re: [HACKERS] statistics for array types

2015-08-24 Thread Jeff Janes
On Thu, Aug 20, 2015 at 6:00 PM, Tomas Vondra wrote: > Hi, > > On 08/11/2015 04:38 PM, Jeff Janes wrote: > >> When reviewing some recent patches, I decided the statistics gathered >> for arrays had some pre-existing shortcomings. >> >> The main one is that w

<    1   2   3   4   5   6   7   8   9   10   >