Re: For standby pg_ctl doesn't wait for PM_STATUS_READY in presence of promote_trigger_file

2020-08-04 Thread Kyotaro Horiguchi
Hello. At Tue, 4 Aug 2020 12:01:45 -0700, Ashwin Agrawal wrote in > If shutdown (non hot enabled) standby and promote the standby using > promote_trigger_file via pg_ctl start with -w (wait), currently pg_ctl > returns as soon as recovery is started. Instead would be helpful if > pg_ctl can

Re: pg13dev: explain partial, parallel hashagg, and memory use

2020-08-04 Thread David Rowley
On Wed, 5 Aug 2020 at 14:27, David Rowley wrote: > > On Wed, 5 Aug 2020 at 14:13, Justin Pryzby wrote: > > Also odd (to me). If I encourage more workers, there are "slots" for each > > "planned" worker, even though fewer were launched: > > Looking at explain.c for "num_workers; " (including the

Re: LSM tree for Postgres

2020-08-04 Thread Peter Geoghegan
On Tue, Aug 4, 2020 at 8:24 AM Alexander Korotkov wrote: > So, in my point of view LSM as an index AM is far not a full power LSM > for PostgreSQL, but it's still useful. Large insert-only tables can > benefit from LSM. Large tables with many indexes could also benefit, > because non-HOT

Re: pg13dev: explain partial, parallel hashagg, and memory use

2020-08-04 Thread David Rowley
On Wed, 5 Aug 2020 at 14:13, Justin Pryzby wrote: > Also odd (to me). If I encourage more workers, there are "slots" for each > "planned" worker, even though fewer were launched: Looking at explain.c for "num_workers; " (including the final space at the end), looking at each for loop that loops

Re: [DOC] Document concurrent index builds waiting on each other

2020-08-04 Thread Alvaro Herrera
On 2020-Aug-04, Alvaro Herrera wrote: > diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h > index b20e2ad4f6..43c8ea3e31 100644 > --- a/src/include/storage/proc.h > +++ b/src/include/storage/proc.h > @@ -53,6 +53,8 @@ struct XidCache > #define PROC_IS_AUTOVACUUM

Re: pg13dev: explain partial, parallel hashagg, and memory use

2020-08-04 Thread Justin Pryzby
On Wed, Aug 05, 2020 at 01:44:17PM +1200, David Rowley wrote: > On Wed, 5 Aug 2020 at 13:21, Justin Pryzby wrote: > > > > I'm testing with a customer's data on pg13dev and got output for which Peak > > Memory doesn't look right/useful. I reproduced it on 565f16902. > > Likely the sanity of

Re: [DOC] Document concurrent index builds waiting on each other

2020-08-04 Thread Alvaro Herrera
On 2020-Mar-25, Andres Freund wrote: > What I was thinking of was a new flag, with a distinct value from > PROC_IN_VACUUM. It'd currently just be specified in the > GetCurrentVirtualXIDs() calls in WaitForOlderSnapshots(). That'd avoid > needing to wait for other CICs on different relations.

Re: pg13dev: explain partial, parallel hashagg, and memory use

2020-08-04 Thread James Coleman
On Tue, Aug 4, 2020 at 9:44 PM David Rowley wrote: > > On Wed, 5 Aug 2020 at 13:21, Justin Pryzby wrote: > > > > I'm testing with a customer's data on pg13dev and got output for which Peak > > Memory doesn't look right/useful. I reproduced it on 565f16902. > > Likely the sanity of those results

Re: pg13dev: explain partial, parallel hashagg, and memory use

2020-08-04 Thread David Rowley
On Wed, 5 Aug 2020 at 13:21, Justin Pryzby wrote: > > I'm testing with a customer's data on pg13dev and got output for which Peak > Memory doesn't look right/useful. I reproduced it on 565f16902. Likely the sanity of those results depends on whether you think that the Memory Usage reported

pg13dev: explain partial, parallel hashagg, and memory use

2020-08-04 Thread Justin Pryzby
I'm testing with a customer's data on pg13dev and got output for which Peak Memory doesn't look right/useful. I reproduced it on 565f16902. CREATE TABLE p(i int) PARTITION BY RANGE(i); CREATE TABLE p1 PARTITION OF p FOR VALUES FROM (0)TO(1000); CREATE TABLE p2 PARTITION OF p FOR VALUES FROM

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-04 Thread Amit Langote
On Wed, Aug 5, 2020 at 10:04 AM Justin Pryzby wrote: > On Wed, Aug 05, 2020 at 09:53:44AM +0900, Amit Langote wrote: > > On Wed, Aug 5, 2020 at 9:52 AM Amit Langote wrote: > > > On Wed, Aug 5, 2020 at 9:32 AM Justin Pryzby wrote: > > > > On Wed, Aug 05, 2020 at 09:26:20AM +0900, Amit Langote

Re: new heapcheck contrib module

2020-08-04 Thread Peter Geoghegan
On Tue, Aug 4, 2020 at 9:44 AM Robert Haas wrote: > I think it might be possible to distinguish between different types of > corruption and to separate, at least to some degree, the checking > associated with each type. I think one can imagine something that > checks the structure of a btree

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-04 Thread Justin Pryzby
On Wed, Aug 05, 2020 at 09:53:44AM +0900, Amit Langote wrote: > On Wed, Aug 5, 2020 at 9:52 AM Amit Langote wrote: > > On Wed, Aug 5, 2020 at 9:32 AM Justin Pryzby wrote: > > > On Wed, Aug 05, 2020 at 09:26:20AM +0900, Amit Langote wrote: > > > > On Wed, Aug 5, 2020 at 12:11 AM Justin Pryzby >

Re: Add MAIN_RELATION_CLEANUP and SECONDARY_RELATION_CLEANUP options to VACUUM

2020-08-04 Thread Bossart, Nathan
On 8/2/20, 11:47 PM, "Michael Paquier" wrote: > + VACOPT_TOAST_CLEANUP = 1 << 6, /* process TOAST table, if any */ > + VACOPT_DISABLE_PAGE_SKIPPING = 1 << 7, /* don't skip any pages */ > + VACOPT_MAIN_REL_CLEANUP = 1 << 8/* process main relation */ > } VacuumOption; > > Do we

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-04 Thread Amit Langote
On Wed, Aug 5, 2020 at 9:52 AM Amit Langote wrote: > On Wed, Aug 5, 2020 at 9:32 AM Justin Pryzby wrote: > > On Wed, Aug 05, 2020 at 09:26:20AM +0900, Amit Langote wrote: > > > On Wed, Aug 5, 2020 at 12:11 AM Justin Pryzby > > > wrote: > > > > > > > > On Tue, Aug 04, 2020 at 08:12:10PM +0900,

RE: Creating foreign key on partitioned table is too slow

2020-08-04 Thread kato-...@fujitsu.com
On Tuesday, July 14, 2020 11:29 PM, Daniel Fustafsson wrote: >Are you able to test Alvaros latest patch to see if that solves the originally >reported problem, so that we can reach >closure on this item during the >commitfest? Sorry for the too late replay. I missed this mail. And, thanks for

Re: LSM tree for Postgres

2020-08-04 Thread Alexander Korotkov
On Tue, Aug 4, 2020 at 7:56 PM Konstantin Knizhnik wrote: > I do not understand why do we need multiple indexes. > We need one "hot" index which fits in memory to perform fast inserts. > But it should not be too small to be able to accumulate substantial > amount of records to provide efficient

Re: Concurrency bug in amcheck

2020-08-04 Thread Peter Geoghegan
On Tue, Aug 4, 2020 at 4:18 PM Alexander Korotkov wrote: > Pushed. Comment is changed as you suggested. But I've replaced "last > pivot tuple" with "remaining tuples", because the page can also have a > high key, which is also a tuple. You're right, of course. Thanks again -- Peter Geoghegan

Re: Concurrency bug in amcheck

2020-08-04 Thread Alexander Korotkov
On Wed, Aug 5, 2020 at 1:58 AM Peter Geoghegan wrote: > On Tue, Aug 4, 2020 at 7:27 AM Alexander Korotkov > wrote: > > Thank you for your reminder. Revised patch is attached. Now, the > > contents of deleted btree pages isn't masked. I've checked that > > installcheck passes with

Re: Concurrency bug in amcheck

2020-08-04 Thread Peter Geoghegan
Hi Alexander, On Tue, Aug 4, 2020 at 7:27 AM Alexander Korotkov wrote: > Thank you for your reminder. Revised patch is attached. Now, the contents > of deleted btree pages isn't masked. I've checked that installcheck passes > with wal_consistency_checking='Btree'. I'm going to push this

Re: Confusing behavior of create table like

2020-08-04 Thread Tom Lane
Peter Eisentraut writes: > On 2020-08-04 19:36, Konstantin Knizhnik wrote: >> Yes, generated columns are also using implicitly generated sequences. >> So them are  very similar with SERIAL/BIGSERIAL columns. This actually >> make we wonder why we can not handle them in the same way in >> CREATE

Re: [patch] demote

2020-08-04 Thread Jehan-Guillaume de Rorthais
Hi, Yet another summary + patch + tests. Demote now keeps backends with no active xid alive. Smart mode keeps all backends: it waits for them to finish their xact and enter read-only. Fast mode terminate backends wit an active xid and keeps all other ones. Backends enters "read-only" using

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-08-04 Thread Tom Lane
Peter Geoghegan writes: > On Tue, Aug 4, 2020 at 1:31 PM Tom Lane wrote: >> while the INSERT_IN_PROGRESS case has none. Simple symmetry >> would suggest that the INSERT_IN_PROGRESS case should be >> >> if (checking_uniqueness || >> HeapTupleIsHeapOnly(heapTuple)) >> //

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-08-04 Thread Peter Geoghegan
On Tue, Aug 4, 2020 at 1:31 PM Tom Lane wrote: > The core of the issue seems to be that in the presence of concurrent > updates, we might not have a stable opinion of which entry of the HOT > chain is live, leading to trying to index multiple ones of them (using > the same root TID), which leads

Re: Confusing behavior of create table like

2020-08-04 Thread Peter Eisentraut
On 2020-08-04 19:36, Konstantin Knizhnik wrote: Yes, generated columns are also using implicitly generated sequences. So them are  very similar with SERIAL/BIGSERIAL columns. This actually make we wonder why we can not handle them in the same way in CREATE TABLE LIKE. The current specification

Re: Support for NSS as a libpq TLS backend

2020-08-04 Thread Daniel Gustafsson
> On 3 Aug 2020, at 21:18, Andrew Dunstan > wrote: > On 8/3/20 12:46 PM, Andrew Dunstan wrote: >> On 7/31/20 4:44 PM, Andrew Dunstan wrote: >>> OK, here is an update of your patch that compiles and runs against NSS >>> under Windows (VS2019). Out of curiosity since I'm not familiar with

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-08-04 Thread Tom Lane
Peter Geoghegan writes: > Any further thoughts on this, Tom? Sorry for slow response ... my ISP had an equipment failure that took out my email service for most of a day. > This is clearly a live bug that we should fix before too long. I could > write the patch myself, but I would like to get

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-04 Thread Robert Haas
On Mon, Aug 3, 2020 at 12:11 PM Justin Pryzby wrote: > (gdb) p *pinfo->relid_map@414 > (gdb) p *partdesc->oids@415 Whoa, I didn't know about using @ in gdb to print multiple elements. Wild! Anyway, these two arrays differ in that the latter array has 22653702 inserted at the beginning and

Re: display offset along with block number in vacuum errors

2020-08-04 Thread Mahendra Singh Thalor
Thanks Sawada and Justin. On Sun, 2 Aug 2020 at 09:33, Masahiko Sawada wrote: > > On Sat, 1 Aug 2020 at 16:02, Mahendra Singh Thalor wrote: > > > > Thanks Justin. > > > > On Sat, 1 Aug 2020 at 11:47, Justin Pryzby wrote: > > > > > > On Fri, Jul 31, 2020 at 04:55:14PM -0500, Justin Pryzby

For standby pg_ctl doesn't wait for PM_STATUS_READY in presence of promote_trigger_file

2020-08-04 Thread Ashwin Agrawal
If shutdown (non hot enabled) standby and promote the standby using promote_trigger_file via pg_ctl start with -w (wait), currently pg_ctl returns as soon as recovery is started. Instead would be helpful if pg_ctl can wait till PM_STATUS_READY for this case, given promotion is requested. pg_ctl

Re: LSM tree for Postgres

2020-08-04 Thread Konstantin Knizhnik
On 04.08.2020 20:44, Tomas Vondra wrote: Unique indexes are not supported now. And I do not see some acceptable solution here. If we will have to check presence of duplicate at the time of insert then it will eliminate all advantages of LSM approach. And if we postpone to the moment of

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2020-08-04 Thread Robert Haas
On Mon, Aug 3, 2020 at 7:49 PM Alvaro Herrera wrote: > Why two transactions? The reason is that in order for this to work, we > make a catalog change (mark it detached), and commit so that all > concurrent transactions can see the change. A second transaction waits > for anybody who holds any

Re: LSM tree for Postgres

2020-08-04 Thread Tomas Vondra
On Tue, Aug 04, 2020 at 08:18:01PM +0300, Konstantin Knizhnik wrote: On 04.08.2020 18:11, Tomas Vondra wrote: On Tue, Aug 04, 2020 at 11:22:13AM +0300, Konstantin Knizhnik wrote: Hi hackers, I want to share results of my last research of implementing LSM index in Postgres. Most of modern

Re: Confusing behavior of create table like

2020-08-04 Thread Konstantin Knizhnik
On 04.08.2020 19:53, Robert Haas wrote: On Mon, Aug 3, 2020 at 12:35 PM Peter Eisentraut wrote: On 2020-08-03 14:58, Konstantin Knizhnik wrote: May be generated and identity columns are good things. I have nothing against them. But what preventing us from providing the similar behavior for

Re: Yet another fast GiST build (typo)

2020-08-04 Thread Andrey M. Borodin
> 30 июля 2020 г., в 06:26, Thomas Munro написал(а): > > On Fri, Jul 10, 2020 at 6:55 PM Andrey M. Borodin > wrote: >> Thanks! Fixed. > > It's not a bug, but I think those 64 bit constants should be wrapped > in UINT64CONST(), following our convention. Thanks, fixed! > I'm confused about

Re: LSM tree for Postgres

2020-08-04 Thread Konstantin Knizhnik
On 04.08.2020 18:18, Stephen Frost wrote: Independently while considering this, I don't think the issues around how to deal with unique btrees properly has really been considered- you certainly can't stop your search on the first tuple you find even if the index is unique, since the "unique"

Re: LSM tree for Postgres

2020-08-04 Thread Konstantin Knizhnik
On 04.08.2020 18:11, Tomas Vondra wrote: On Tue, Aug 04, 2020 at 11:22:13AM +0300, Konstantin Knizhnik wrote: Hi hackers, I want to share results of my last research of implementing LSM index in Postgres. Most of modern databases (RocksDB, MongoDB, Tarantool,...) are using LSM tree

Re: LSM tree for Postgres

2020-08-04 Thread Konstantin Knizhnik
On 04.08.2020 18:04, Alexander Korotkov wrote: Hi! On Tue, Aug 4, 2020 at 11:22 AM Konstantin Knizhnik wrote: I want to share results of my last research of implementing LSM index in Postgres. Most of modern databases (RocksDB, MongoDB, Tarantool,...) are using LSM tree instead of

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2020-08-04 Thread Alvaro Herrera
On 2020-Aug-03, Alvaro Herrera wrote: > Why two transactions? The reason is that in order for this to work, we > make a catalog change (mark it detached), and commit so that all > concurrent transactions can see the change. A second transaction waits > for anybody who holds any lock on the

Re: Confusing behavior of create table like

2020-08-04 Thread Robert Haas
On Mon, Aug 3, 2020 at 12:35 PM Peter Eisentraut wrote: > On 2020-08-03 14:58, Konstantin Knizhnik wrote: > > May be generated and identity columns are good things. I have nothing > > against them. > > But what preventing us from providing the similar behavior for > > serial/bigseries types? > >

Re: Can a background worker exist without shared memory access for EXEC_BACKEND cases?

2020-08-04 Thread Robert Haas
On Tue, Aug 4, 2020 at 7:27 AM Bharath Rupireddy wrote: > I could get these points earlier in my initial analysis. In fact, I > could figure out the flow on Windows, how these parameters are shared > using a shared file(CreateFileMapping(), MapViewOfFile()), and the > shared file name being

Re: new heapcheck contrib module

2020-08-04 Thread Robert Haas
On Tue, Aug 4, 2020 at 12:00 PM Peter Geoghegan wrote: > With indexes you tend to have redundancy in how relationships among > pages are described. So you have siblings whose pointers must be in > agreement (left points to right, right points to left), and it's not > clear which one you should

Re: Amcheck: do rightlink verification with lock coupling

2020-08-04 Thread Andrey M. Borodin
> 4 авг. 2020 г., в 03:44, Peter Geoghegan написал(а): > > On Mon, Jul 20, 2020 at 11:46 AM Andrey M. Borodin > wrote: >> In this thread [0] we decided that lock coupling is necessary for >> btree_xlog_unlink_page(). >> So, maybe let's revive this patch? > > Yes, let's do that. Can you

Re: Parallel copy

2020-08-04 Thread Tomas Vondra
On Mon, Aug 03, 2020 at 12:33:48PM +0530, Bharath Rupireddy wrote: On Sat, Aug 1, 2020 at 9:55 AM vignesh C wrote: The patches were not applying because of the recent commits. I have rebased the patch over head & attached. I rebased v2-0006-Parallel-Copy-For-Binary-Format-Files.patch.

Re: Error on failed COMMIT

2020-08-04 Thread Dave Cramer
Attached is the rebased patch for consideration. Dave Cramer www.postgres.rocks > > 0001-Throw-error-and-rollback-on-a-failed-transaction-ins.patch Description: Binary data

Re: new heapcheck contrib module

2020-08-04 Thread Robert Haas
On Fri, Jul 31, 2020 at 12:33 PM Andres Freund wrote: > I'm not sure what I was thinking "back then", but right now I'd argue > that the best lock against vacuum isn't a SUE, but announcing the > correct ->xmin, so you can be sure that clog entries won't be yanked out > from under you.

Re: new heapcheck contrib module

2020-08-04 Thread Peter Geoghegan
On Tue, Aug 4, 2020 at 7:59 AM Robert Haas wrote: > I think we should try not to imagine anything in particular. Just to > be clear, I am not trying to knock what you have; I know it was a lot > of work to create and it's a huge improvement over having nothing. But > in my mind, a perfect tool

Re: LSM tree for Postgres

2020-08-04 Thread Alexander Korotkov
On Tue, Aug 4, 2020 at 6:11 PM Tomas Vondra wrote: > On Tue, Aug 04, 2020 at 11:22:13AM +0300, Konstantin Knizhnik wrote: > >Hi hackers, > > > >I want to share results of my last research of implementing LSM index > >in Postgres. > >Most of modern databases (RocksDB, MongoDB, Tarantool,...) are

Re: WIP: BRIN multi-range indexes

2020-08-04 Thread Tomas Vondra
On Tue, Aug 04, 2020 at 05:36:51PM +0300, Alexander Korotkov wrote: Hi, Tomas! Sorry for the late reply. On Sun, Jul 19, 2020 at 6:19 PM Tomas Vondra wrote: I think there's a number of weak points in this approach. Firstly, it assumes the summaries can be represented as arrays of built-in

Re: LSM tree for Postgres

2020-08-04 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On Tue, Aug 04, 2020 at 11:22:13AM +0300, Konstantin Knizhnik wrote: > >two top indexes used in cyclic way and one main index. When top index > >reaches some threshold value > >we initiate merge with main index, done by bgworker

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-04 Thread Justin Pryzby
On Tue, Aug 04, 2020 at 08:12:10PM +0900, Amit Langote wrote: > It may be this commit that went into PG 12 that is causing the problem: Thanks for digging into this. > to account for partitions that were pruned by the planner for which we > decided to put 0 into relid_map, but it only considered

Re: LSM tree for Postgres

2020-08-04 Thread Tomas Vondra
On Tue, Aug 04, 2020 at 11:22:13AM +0300, Konstantin Knizhnik wrote: Hi hackers, I want to share results of my last research of implementing LSM index in Postgres. Most of modern databases (RocksDB, MongoDB, Tarantool,...) are using LSM tree instead of classical B-Tree. I was under the

Re: LSM tree for Postgres

2020-08-04 Thread Alexander Korotkov
Hi! On Tue, Aug 4, 2020 at 11:22 AM Konstantin Knizhnik wrote: > I want to share results of my last research of implementing LSM index in > Postgres. > Most of modern databases (RocksDB, MongoDB, Tarantool,...) are using LSM tree > instead of classical B-Tree. I wouldn't say it that way. I

Re: new heapcheck contrib module

2020-08-04 Thread Robert Haas
On Mon, Aug 3, 2020 at 1:16 PM Peter Geoghegan wrote: > If you really wanted to do this, > you'd have to describe a practical scenario under which it made sense > to soldier on, where we'd definitely be able to count the number of > problems in a meaningful way, without much risk of either

Re: WIP: BRIN multi-range indexes

2020-08-04 Thread Alexander Korotkov
Hi, Tomas! Sorry for the late reply. On Sun, Jul 19, 2020 at 6:19 PM Tomas Vondra wrote: > I think there's a number of weak points in this approach. > > Firstly, it assumes the summaries can be represented as arrays of > built-in types, which I'm not really sure about. It clearly is not true >

Re: Concurrency bug in amcheck

2020-08-04 Thread Alexander Korotkov
Hi, Peter! On Sat, Aug 1, 2020 at 3:23 AM Peter Geoghegan wrote: > On Wed, May 13, 2020 at 4:06 PM Peter Geoghegan wrote: > > On Mon, May 11, 2020 at 5:56 AM Alexander Korotkov > > wrote: > > > Thank you. 2nd patch is proposed for master and makes btree page > > > unlink remove all the items

Re: Since '2001-09-09 01:46:40'::timestamp microseconds are lost when extracting epoch

2020-08-04 Thread Pavel Stehule
út 4. 8. 2020 v 16:08 odesílatel Peter Eisentraut < peter.eisentr...@2ndquadrant.com> napsal: > On 2020-05-25 15:28, Peter Eisentraut wrote: > > On 2019-12-02 23:52, Thomas Munro wrote: > >>> I'm not an expert in floating point math but hopefully it means that no > >>> type change is required -

Re: Since '2001-09-09 01:46:40'::timestamp microseconds are lost when extracting epoch

2020-08-04 Thread Peter Eisentraut
On 2020-05-25 15:28, Peter Eisentraut wrote: On 2019-12-02 23:52, Thomas Munro wrote: I'm not an expert in floating point math but hopefully it means that no type change is required - double precision can handle it. Me neither, but the SQL standard requires us to use an exact numeric type, so

Re: Replace remaining StrNCpy() by strlcpy()

2020-08-04 Thread Peter Eisentraut
On 2020-08-03 19:39, Tom Lane wrote: That's easy to fix, but it's perhaps wondering briefly why it needs to be zero-padded. hashname() doesn't care, heap_form_tuple() doesn't care. Does anything care? We do have an expectation that there are no undefined bytes in values to be stored on-disk.

Re: Reduce/eliminate the impact of FPW

2020-08-04 Thread Stephen Frost
Greetings, Please don't top-post on these lists. * SATYANARAYANA NARLAPURAM (satyanarlapu...@gmail.com) wrote: > Increasing checkpoint_timeout helps reduce the amount of log written to the > disk. This has several benefits like, reduced number of WAL IO, archival > load on the system, less

Yet another issue with step generation in partition pruning

2020-08-04 Thread Etsuro Fujita
Commit 13838740f fixed some issues with step generation in partition pruning, but as I mentioned in [1], I noticed that there is yet another issue: get_steps_using_prefix() assumes that clauses in the passed-in prefix list are sorted in ascending order of their partition key numbers, but the

Re: problem with RETURNING and update row movement

2020-08-04 Thread Etsuro Fujita
Amit-san, On Mon, Aug 3, 2020 at 2:55 PM Amit Langote wrote: > However, I noticed that having system columns like ctid, xmin, etc. in > the RETURNING list is now broken and maybe irrepairably due to the > approach we are taking in the patch. Let me show an example: > > drop table foo; > create

Re: Can a background worker exist without shared memory access for EXEC_BACKEND cases?

2020-08-04 Thread Bharath Rupireddy
Thank you Robert for the comments. On Mon, Aug 3, 2020 at 9:19 PM Robert Haas wrote: > > On Fri, Jul 31, 2020 at 11:13 PM Bharath Rupireddy > wrote: > > memory. MyLatch variable also gets created in shared mode. And having > > no shared memory access for the worker for EXEC_BACKEND cases(in > >

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-04 Thread Amit Langote
On Tue, Aug 4, 2020 at 1:11 AM Justin Pryzby wrote: > > On Mon, Aug 03, 2020 at 11:41:37AM -0400, Robert Haas wrote: > > On Sun, Aug 2, 2020 at 2:11 PM Justin Pryzby wrote: > > > Based on commit logs, I suspect this may be an "older bug", specifically > > > maybe > > > with: > > > > > > |commit

Re: EDB builds Postgres 13 with an obsolete ICU version

2020-08-04 Thread Dave Page
On Tue, Aug 4, 2020 at 10:29 AM Magnus Hagander wrote: > On Tue, Aug 4, 2020 at 10:07 AM Dave Page wrote: > >> >> >> On Tue, Aug 4, 2020 at 1:04 AM Bruce Momjian wrote: >> >>> On Mon, Aug 3, 2020 at 08:56:06PM +0200, Daniel Verite wrote: >>> > Hi, >>> > >>> > As a follow-up to bug #16570 [1]

Re: EDB builds Postgres 13 with an obsolete ICU version

2020-08-04 Thread Magnus Hagander
On Tue, Aug 4, 2020 at 10:07 AM Dave Page wrote: > > > On Tue, Aug 4, 2020 at 1:04 AM Bruce Momjian wrote: > >> On Mon, Aug 3, 2020 at 08:56:06PM +0200, Daniel Verite wrote: >> > Hi, >> > >> > As a follow-up to bug #16570 [1] and other previous discussions >> > on the mailing-lists, I'm

Re: EDB builds Postgres 13 with an obsolete ICU version

2020-08-04 Thread Thomas Kellerer
Dave Page schrieb am 04.08.2020 um 10:06: > Correct - updating ICU would mean a reindex is required following any > upgrade, major or minor. > > I would really like to find an acceptable solution to this however as > it really would be good to be able to update ICU. > What about providing a newer

Re: [PATCH] - Provide robust alternatives for replace_string

2020-08-04 Thread Asim Praveen
> On 03-Aug-2020, at 8:36 PM, Alvaro Herrera wrote: > > On 2020-Aug-03, Asim Praveen wrote: > >> Thank you Alvaro for reviewing the patch! >> >>> On 01-Aug-2020, at 7:22 AM, Alvaro Herrera wrote: >>> >>> What happens if a replacement string happens to be split in the middle >>> by the

Re: [PATCH v1] elog.c: Remove special case which avoided %*s format strings..

2020-08-04 Thread David Rowley
On Tue, 4 Aug 2020 at 19:36, Michael Paquier wrote: > Did you check if our implementation of src/port/snprintf.c makes %*s > much slower than %s or not? FWIW, I have just run a small test on my > laptop, and running 100M calls of snprintf() with "%s" in a tight loop > takes 2.7s, with "%*s" and

Re: SSL TAP test fails due to default client certs.

2020-08-04 Thread Kyotaro Horiguchi
At Tue, 4 Aug 2020 14:43:58 +0900, Michael Paquier wrote in > On Tue, Aug 04, 2020 at 12:00:33PM +0900, Kyotaro Horiguchi wrote: > > While poking at ssl code, I noticed that 002_scram.pl fails if > > ~/.postgresql/root.crt exists. This has been fixed once but > > d6e612f837 reintroduced one.

Re: Is it worth accepting multiple CRLs?

2020-08-04 Thread Kyotaro Horiguchi
At Mon, 03 Aug 2020 16:20:40 +0900 (JST), Kyotaro Horiguchi wrote in > Thanks for the opinion. I'll continue working on this. This is it, but.. Looking closer I realized that certificates are verified in each backend so CRL cache doesn't work at all for the hashed directory method.

Rewrite view?

2020-08-04 Thread Hao Wu
Hi hackers, When I define a view relevant with `char varying`, it shows the type is casted to text. See, gpadmin=# CREATE TABLE foobar (a character varying); CREATE TABLE gpadmin=# CREATE VIEW fooview AS SELECT * FROM foobar WHERE a::character varying = ANY(ARRAY['foo'::character varying,

LSM tree for Postgres

2020-08-04 Thread Konstantin Knizhnik
Hi hackers, I want to share results of my last research of implementing LSM index in Postgres. Most of modern databases (RocksDB, MongoDB, Tarantool,...) are using LSM tree instead of classical B-Tree. From one side, capacity of RAM at modern servers allows to keep the whole database in

Re: EDB builds Postgres 13 with an obsolete ICU version

2020-08-04 Thread Dave Page
On Tue, Aug 4, 2020 at 1:04 AM Bruce Momjian wrote: > On Mon, Aug 3, 2020 at 08:56:06PM +0200, Daniel Verite wrote: > > Hi, > > > > As a follow-up to bug #16570 [1] and other previous discussions > > on the mailing-lists, I'm checking out PG13 beta for Windows > > from: > >

Re: [PATCH v1] elog.c: Remove special case which avoided %*s format strings..

2020-08-04 Thread Michael Paquier
On Sun, Aug 02, 2020 at 11:59:48PM -0500, Justin Pryzby wrote: > ..which should no longer be needed since it was a performance hack for > specific > platform snprintf, which are no longer used. Did you check if our implementation of src/port/snprintf.c makes %*s much slower than %s or not?

Re: ModifyTable overheads in generic plans

2020-08-04 Thread Amit Langote
On Sat, Aug 1, 2020 at 4:46 AM Robert Haas wrote: > On Fri, Jun 26, 2020 at 8:36 AM Amit Langote wrote: > > 0001 and 0002 are preparatory patches. > > I read through these patches a bit but it's really unclear what the > point of them is. I think they need better commit messages, or better >

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

2020-08-04 Thread Takashi Menjo
Dear hackers, I rebased my old patchset. It would be good to compare this v4 patchset to non-volatile WAL buffer's one [1]. [1] https://www.postgresql.org/message-id/002101d649fb$1f5966e0$5e0c34a0$@hco.ntt.co.jp_1 Regards, Takashi -- Takashi Menjo

Re: Handing off SLRU fsyncs to the checkpointer

2020-08-04 Thread Thomas Munro
On Wed, Feb 12, 2020 at 9:54 PM Thomas Munro wrote: > In commit 3eb77eba we made it possible for any subsystem that wants a > file to be flushed as part of the next checkpoint to ask the > checkpointer to do that, as previously only md.c could do. Hello, While working on recovery performance, I