Re: Planner chose a much slower plan in hashjoin, using a large table as the inner table.

2019-11-18 Thread Thomas Munro
On Mon, Nov 18, 2019 at 7:48 PM Jinbao Chen wrote: > In the test case above, the small table has 3000 tuples and 100 distinct > values on column ‘a’. > If we use small table as inner table. The chan length of the bucket is 30. > And we need to > search the whole chain on probing the hash

Re: Hypothetical indexes using BRIN broken since pg10

2019-11-18 Thread Julien Rouhaud
On Tue, Nov 19, 2019 at 6:40 AM Michael Paquier wrote: > > On Fri, Nov 15, 2019 at 12:07:15PM +0900, Michael Paquier wrote: > > So, Heikki, are you planning to work more on that and commit a change > > close to what has been proposed upthread in [1]? It sounds to me that > > this has the

RE: Recovery performance of DROP DATABASE with many tablespaces

2019-11-18 Thread k.jami...@fujitsu.com
On Wed, Nov 13, 2019 5:34PM (GMT+9), Fujii Masao wrote: > On Wed, Nov 13, 2019 at 3:57 PM k.jami...@fujitsu.com > wrote: > > > > On Wed, Oct. 2, 2019 5:40 PM, Fujii Masao wrote: > > > On Tue, Jul 10, 2018 at 3:04 PM Michael Paquier > wrote: > > > > > > > > On Thu, Jul 05, 2018 at 01:42:20AM

Re: Hypothetical indexes using BRIN broken since pg10

2019-11-18 Thread Michael Paquier
On Fri, Nov 15, 2019 at 12:07:15PM +0900, Michael Paquier wrote: > So, Heikki, are you planning to work more on that and commit a change > close to what has been proposed upthread in [1]? It sounds to me that > this has the advantage to be non-intrusive and a similar solution has > been used for

Re: [PATCH] Implement INSERT SET syntax

2019-11-18 Thread Gareth Palmer
> On 15/11/2019, at 10:20 AM, Tom Lane wrote: > > Gareth Palmer writes: >>> On 19/08/2019, at 3:00 AM, Tom Lane wrote: >>> Perhaps the way to resolve Peter's objection is to make the syntax >>> more fully like UPDATE: >>>INSERT INTO target SET c1 = x, c2 = y+z, ... FROM

Re: Invisible PROMPT2

2019-11-18 Thread Thomas Munro
On Tue, Nov 19, 2019 at 12:09 PM Tom Lane wrote: > You should follow the logic in pg_wcswidth: compute PQmblen() first, > and bail out if it's more than the remaining string length, otherwise > it's ok to apply PQdsplen(). Got it. I was worried that it wasn't safe to call even PQmblen(),

Re: [PATCH] Implement INSERT SET syntax

2019-11-18 Thread Gareth Palmer
> On 15/11/2019, at 10:20 AM, Tom Lane wrote: > > Gareth Palmer writes: >>> On 19/08/2019, at 3:00 AM, Tom Lane wrote: >>> Perhaps the way to resolve Peter's objection is to make the syntax >>> more fully like UPDATE: >>>INSERT INTO target SET c1 = x, c2 = y+z, ... FROM

Re: progress report for ANALYZE

2019-11-18 Thread Amit Langote
Yamada-san, Thanks for working on this. On Wed, Nov 6, 2019 at 2:50 PM Tatsuro Yamada wrote: > I revised the patch as following because I realized counting the types of ext > stats is not useful for users. > > - Attached new patch counts a number of ext stats instead the types of ext >

Re: PITR on DROP DATABASE, deleting of the database directory despite the recovery_target_time set before.

2019-11-18 Thread Craig Ringer
On Mon, 18 Nov 2019 at 18:48, Nicolas Lutic wrote: > Dear Hackers, > > After a drop database > with FORCE? > , he tried to recover the data on the last inserted transaction by using > the recovery_target_time. > The issue is the database is present in the system catalog but the > directory

Re: physical slot xmin dependency on logical slot?

2019-11-18 Thread Craig Ringer
On Tue, 19 Nov 2019 at 05:37, Jeremy Finzel wrote: > We had a scenario today that was new to us. We had a logical replication > slot that was severely far behind. Before dropping this logical slot, we > made a physical point-in-time-recovery snapshot of the system with this > logical slot. > >

Re: Postgres on IBM z/OS 2.2.0 and 2.3.0

2019-11-18 Thread Tom Lane
parveen mehta writes: > Thanks for providing valuable inputs into my concerns. In the last part you > mentioned and i am quoting it here "would limit the parts of the system > that would have to be cleansed of ASCII-isms to libpq and src/bin/. > But that's already a nontrivial headache I

Re: Duplicate Workers entries in some EXPLAIN plans

2019-11-18 Thread Maciek Sakrejda
On Thu, Oct 24, 2019 at 6:48 PM Andres Freund wrote: > Unfortunately I think the fix isn't all that trivial, due to the way we > output the per-worker information at the end of ExplainNode(), by just > dumping things into a string. It seems to me that a step in the right > direction would be for

Re: Invisible PROMPT2

2019-11-18 Thread Tom Lane
Thomas Munro writes: > Right, a PQdsplen()/PQmblen() loop works nicely, as attached. > I spotted a potential problem: I suppose I could write a PROMPT1 that > includes an invalid multibyte sequence at the end of the buffer and > trick PQmblen() or PQdsplen() into reading a few bytes past the

Re: physical slot xmin dependency on logical slot?

2019-11-18 Thread Andres Freund
Hi, On 2019-11-18 15:36:47 -0600, Jeremy Finzel wrote: > We had a scenario today that was new to us. We had a logical replication > slot that was severely far behind. Before dropping this logical slot, we > made a physical point-in-time-recovery snapshot of the system with this > logical slot.

Re: Reverse collations (initially for making keyset pagination cover more cases)

2019-11-18 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> Well, one obvious completely general method is to teach the planner >> (somehow) to spot conditions of the form >> (a > $1 OR (a = $1 AND b > $2) OR (a = $1 AND b = $2 AND c > $3) ...) >> etc. and make them indexable if the sense of the > or < operator at

physical slot xmin dependency on logical slot?

2019-11-18 Thread Jeremy Finzel
We had a scenario today that was new to us. We had a logical replication slot that was severely far behind. Before dropping this logical slot, we made a physical point-in-time-recovery snapshot of the system with this logical slot. This logical slot was causing severe catalog bloat. We

Re: Invisible PROMPT2

2019-11-18 Thread Thomas Munro
On Tue, Nov 19, 2019 at 6:21 AM Tom Lane wrote: > Thomas Munro writes: > > Yeah. Maybe pg_wcswidth() would be OK though, and it's available in > > psql, though I guess you'd have to make a copy with the escaped bits > > stripped out. > > Right, you should use pg_wcswidth() or the underlying

Re: segfault in geqo on experimental gcc animal

2019-11-18 Thread Fabien COELHO
Hello Martin, The issue is resolved now and tests are fine for me. I recompiled gcc trunk and the moonjelly is back to green. Thanks! -- Fabien.

Re: Postgres on IBM z/OS 2.2.0 and 2.3.0

2019-11-18 Thread parveen mehta
Tom, Thanks for providing valuable inputs into my concerns. In the last part you mentioned and i am quoting it here "would limit the parts of the system that would have to be cleansed of ASCII-isms to libpq and src/bin/. But that's already a nontrivial headache I suspect." I am not clear on the 

Re: 'Invalid lp' during heap_xlog_delete

2019-11-18 Thread Daniel Wood
> I'll try to look into that with lower page sizes for relation and WAL > pages. The page size is totally unrelated to this bug. When you repro the redo failure it is because the log record is being applied to an old page version. The correct newer page version never got written because

Re: [HACKERS] proposal: schema variables

2019-11-18 Thread Pavel Stehule
_close was replaced by table_open, > table_close > fresh rebase Regards Pavel > Regards > > Pavel > schema-variables-20191118.patch.gz Description: application/gzip

Re: [HACKERS] pg_shmem_allocations view

2019-11-18 Thread Andres Freund
Hi, On 2019-11-18 21:49:55 +0900, Michael Paquier wrote: > +/* SQL SRF showing allocated shared memory */ > +Datum > +pg_get_shmem_allocations(PG_FUNCTION_ARGS) > This could be more talkative. I don't really see what it'd say, except restate the function name as a sentence? I think that kind of

Re: Reverse collations (initially for making keyset pagination cover more cases)

2019-11-18 Thread Tom Lane
Andrew Gierth writes: > "Tom" == Tom Lane writes: > Tom> Lastly, your proposed use-case has some attraction, but this > Tom> proposal only supports it if the column you need to be differently > Tom> sorted is textual. What if the sort columns are all numerics and > Tom> timestamps? > There

Re: [PATCH][BUG FIX] Unsafe access pointers.

2019-11-18 Thread Alvaro Herrera
On 2019-Nov-15, Ranier Vilela wrote: > Hi, > Last time, I promise. > > It's probably not happening, but it can happen, I think. This patch assumes that anything can happen after elog(ERROR). That's wrong -- under ERROR or higher, elog() (as well as ereport) never returns to the caller. If

Re: Invisible PROMPT2

2019-11-18 Thread Tom Lane
Thomas Munro writes: > On Mon, Nov 18, 2019 at 1:49 PM Alvaro Herrera > wrote: >> On 2019-Nov-18, Thomas Munro wrote: >>> Nice idea. Here's one like that, that just does the counting at the >>> end and looks out for readline control codes. It's pretty naive about >>> what "width" means

Re: Role membership and DROP

2019-11-18 Thread Laurenz Albe
On Fri, 2019-11-15 at 13:41 -0500, Tom Lane wrote: > Laurenz Albe writes: > > On Wed, 2019-11-13 at 17:17 -0500, Tom Lane wrote: > > > It might be worth clarifying this point in section 5.7, > > > https://www.postgresql.org/docs/devel/ddl-priv.html > > > but let's not duplicate that in every ref/

Re: function calls optimization

2019-11-18 Thread Andrzej Barszcz
Hi I need advice. ResetExprContext(econtext) is defined as MemoryContextReset((econtext)->ecxt_per_tuple_memory). I can register callback in MemoryContext but it is always cleaned on every call to MemoryContextReset(). How to reset some fields of ExprContext ( living in per_query_memory ) when

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2019-11-18 Thread Amit Khandekar
On Mon, 18 Nov 2019 at 17:52, Amit Kapila wrote: > I have one more question regarding this patch. It seems to me that > the files opened via OpenTransientFile or OpenTemporaryFile are > automatically closed at transaction end(abort), but that doesn't seem > to be the case for files opened with

Re: dropdb --force

2019-11-18 Thread Pavel Stehule
po 18. 11. 2019 v 7:39 odesílatel Pavel Stehule napsal: > > > po 18. 11. 2019 v 7:37 odesílatel Amit Kapila > napsal: > >> On Mon, Nov 18, 2019 at 10:59 AM Pavel Stehule >> wrote: >> > po 18. 11. 2019 v 6:24 odesílatel Amit Kapila >> napsal: >> >> >> >> On Mon, Nov 18, 2019 at 10:33 AM Pavel

Re: 'Invalid lp' during heap_xlog_delete

2019-11-18 Thread Michael Paquier
On Thu, Nov 14, 2019 at 07:38:19PM -0800, Daniel Wood wrote: > Sorry I missed one thing. Turn off full page writes. Hmm. Linux FSes use typically 4kB pages. I'll try to look into that with lower page sizes for relation and WAL pages. > I'm running in an env. with atomic 8K writes. What's

Re: Getting Recordset through returning refcursor

2019-11-18 Thread Michael Paquier
On Fri, Nov 15, 2019 at 03:55:28PM +0530, Amit Kapila wrote: > I think the chances of getting an answer on ODBC related queries will > be more if you post on pgsql-interfaces. There is also a mailing list dedicated to Postgres ODBC: https://www.postgresql.org/list/pgsql-odbc/ -- Michael

Re: [HACKERS] pg_shmem_allocations view

2019-11-18 Thread Michael Paquier
On Fri, Nov 15, 2019 at 11:59:34AM -0800, Andres Freund wrote: > On 2019-11-15 14:43:09 -0500, Robert Haas wrote: >> This never got applied, and that annoyed me again today, so here's a >> new version that I've whacked around somewhat and propose to commit. I >> ripped out the stuff pertaining to

Re: Attempt to consolidate reading of XLOG page

2019-11-18 Thread Michael Paquier
On Fri, Nov 15, 2019 at 06:41:02PM -0300, Alvaro Herrera wrote: > I don't quite understand why you backed off from switching to pread. It > seemed a good change to me. > > [...] > > Having seek/open be a boolean "xlr_seek" seems a bit weird. Changed to > an "operation" enum. (Maybe if we go

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2019-11-18 Thread Amit Kapila
On Mon, Nov 18, 2019 at 5:20 PM Amit Kapila wrote: > > On Thu, Oct 3, 2019 at 4:48 PM Amit Khandekar wrote: > > > > On Wed, 18 Sep 2019 at 12:24, Amit Khandekar wrote: > > > Probably, for now at least, what everyone seems to agree is to take my > > > earlier attached patch forward. > > > > > >

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2019-11-18 Thread Amit Khandekar
On Mon, 18 Nov 2019 at 17:20, Amit Kapila wrote: > I see that you have made changes in ReorderBufferRestoreChanges to use > PathNameOpenFile, but not in ReorderBufferSerializeTXN. Is there a > reason for the same? In my test environment, with the test provided > by you, I got the error

Re: segfault in geqo on experimental gcc animal

2019-11-18 Thread Martin Liška
Hello. The issue is resolved now and tests are fine for me. Martin On Fri, 15 Nov 2019 at 13:11, Martin Liška wrote: > > Heh, it's me who now breaks postgresql build: > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92529 > > Martin > > On Fri, 15 Nov 2019 at 13:01, Fabien COELHO > wrote: > > >

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2019-11-18 Thread Amit Kapila
On Thu, Oct 3, 2019 at 4:48 PM Amit Khandekar wrote: > > On Wed, 18 Sep 2019 at 12:24, Amit Khandekar wrote: > > Probably, for now at least, what everyone seems to agree is to take my > > earlier attached patch forward. > > > > I am going to see if I can add a TAP test for the patch, and will

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-11-18 Thread Dilip Kumar
On Fri, Nov 15, 2019 at 4:19 PM Amit Kapila wrote: > > On Fri, Nov 15, 2019 at 4:01 PM Dilip Kumar wrote: > > > > On Fri, Nov 15, 2019 at 3:50 PM Amit Kapila wrote: > > > > > > > > > Few other comments on this patch: > > > 1. > > > + case REORDER_BUFFER_CHANGE_INVALIDATION: > > > + > > > + /* >

PITR on DROP DATABASE, deleting of the database directory despite the recovery_target_time set before.

2019-11-18 Thread Nicolas Lutic
Dear Hackers, A customer reported a strange behaviour on a PITR restoration.  After a drop database, he tried to recover the data on the last inserted transaction by using the recovery_target_time. The issue is the database is present in the system catalog but the directory was still deleted.

Re: adding partitioned tables to publications

2019-11-18 Thread Amit Langote
On Tue, Nov 12, 2019 at 10:11 AM Amit Langote wrote: > Initial > syncing code can be easily modified to support any combination of > source and target relations, but changes needed for real-time > replication seem non-trivial. I have spent some time hacking on this. With the attached updated

Re: SegFault on 9.6.14

2019-11-18 Thread Amit Kapila
On Fri, Oct 18, 2019 at 10:08 AM Amit Kapila wrote: > > On Thu, Oct 17, 2019 at 10:51 AM Thomas Munro wrote: > > > > === > > Don't shut down Gather[Merge] early under Limit. > > > > Revert part of commit 19df1702f5. > > > > Early shutdown was added by that commit so that we could collect > >