Re: Protect syscache from bloating with negative cache entries

2021-01-27 Thread Kyotaro Horiguchi
At Wed, 27 Jan 2021 13:11:55 +0200, Heikki Linnakangas wrote in > On 27/01/2021 03:13, Kyotaro Horiguchi wrote: > > At Thu, 14 Jan 2021 17:32:27 +0900 (JST), Kyotaro Horiguchi > > wrote in > >> The commit 4656e3d668 (debug_invalidate_system_caches_always) > >> conflicted with this patch.

Re: [PATCH] remove pg_standby

2021-01-27 Thread Michael Paquier
On Wed, Jan 27, 2021 at 05:08:56PM +0900, Fujii Masao wrote: > But one question is; shouldn't we follow "usual" way to retire the > feature instead of dropping that immediately? That is, mark > pg_standby as obsolete, announce that pg_standby will be dropped > after several releases, and then drop

Re: Two patches to speed up pg_rewind.

2021-01-27 Thread Michael Paquier
On Wed, Jan 27, 2021 at 09:18:48AM +, Paul Guo wrote: > Second one is use copy_file_range() for the local rewind case to replace > read()+write(). > This introduces copy_file_range() check and HAVE_COPY_FILE_RANGE so other > code could use copy_file_range() if needed. copy_file_range() was

Re: Add SQL function for SHA1

2021-01-27 Thread Michael Paquier
On Tue, Jan 26, 2021 at 09:53:52PM -0500, Sehrope Sarkuni wrote: > The refactor patch looks good. It builds and passes make check. Thanks for double-checking! The refactoring has been just done as of f854c69. -- Michael signature.asc Description: PGP signature

Re: protect pg_stat_statements_info() for being used without the library loaded

2021-01-27 Thread Michael Paquier
On Thu, Jan 28, 2021 at 03:53:54PM +0900, Fujii Masao wrote: > Thanks! No problem. Applied as of bca96dd. -- Michael signature.asc Description: PGP signature

Re: Wrong usage of RelationNeedsWAL

2021-01-27 Thread Noah Misch
On Thu, Jan 28, 2021 at 12:06:27PM +0900, Kyotaro Horiguchi wrote: > At Wed, 27 Jan 2021 02:48:48 -0800, Noah Misch wrote in > > On Thu, Jan 21, 2021 at 01:23:36AM -0800, Noah Misch wrote: > > > On Thu, Jan 21, 2021 at 06:02:11PM +0900, Kyotaro Horiguchi wrote: > > > > Perhaps I'm missing

Re: Single transaction in the tablesync worker?

2021-01-27 Thread Peter Smith
On Wed, Jan 27, 2021 at 2:53 PM Amit Kapila wrote: > > On Sat, Jan 23, 2021 at 5:56 PM Amit Kapila wrote: > > > > On Sat, Jan 23, 2021 at 4:55 AM Peter Smith wrote: > > > > > > PSA the v18 patch for the Tablesync Solution1. > > > > 7. Have you tested with the new patch the scenario where we

Re: pg_upgrade fails with non-standard ACL

2021-01-27 Thread Noah Misch
On Wed, Jan 27, 2021 at 07:32:42PM +0300, Anastasia Lubennikova wrote: > On 27.01.2021 14:21, Noah Misch wrote: > >On Mon, Jan 25, 2021 at 10:14:43PM +0300, Anastasia Lubennikova wrote: > > > >>1) Could you please clarify, what do you mean by REVOKE failures? > >> > >>I tried following example: >

Re: protect pg_stat_statements_info() for being used without the library loaded

2021-01-27 Thread Fujii Masao
On 2021/01/28 15:42, Michael Paquier wrote: On Thu, Jan 28, 2021 at 08:49:54AM +0800, Julien Rouhaud wrote: Good catch, and patch looks good to me. This crashes the server, cash. Looking at all the other modules in the tree, I am not seeing any other hole. This is new as of 9fbc3f3, and

Re: protect pg_stat_statements_info() for being used without the library loaded

2021-01-27 Thread Michael Paquier
On Thu, Jan 28, 2021 at 08:49:54AM +0800, Julien Rouhaud wrote: > Good catch, and patch looks good to me. This crashes the server, cash. Looking at all the other modules in the tree, I am not seeing any other hole. This is new as of 9fbc3f3, and I will apply it on HEAD. -- Michael

RE: libpq debug log

2021-01-27 Thread k.jami...@fujitsu.com
On Mon, Jan 25, 2021 10:11 PM (JST), Alvaro Herrera wrote: > On 2021-Jan-25, tsunakawa.ta...@fujitsu.com wrote: > > > Iwata-san, > > [...] > > > Considering these and the compilation error Kirk-san found, I'd like > > you to do more self-review before I resume this review. > > Kindly note that

Re: Single transaction in the tablesync worker?

2021-01-27 Thread Peter Smith
Hi Amit. PSA the v21 patch for the Tablesync Solution1. Main differences from v20: + Rebased to latest OSS HEAD @ 27/Jan + v21 is a merging of patches [v17] and [v20], which was made necessary when it was found [ak0127] that the v20 usage of TEMPORARY tablesync slots did not work correctly. v21

Re: Support for NSS as a libpq TLS backend

2021-01-27 Thread Michael Paquier
On Wed, Jan 27, 2021 at 06:47:17PM +, Jacob Champion wrote: > So to check understanding: the `openssl` config variable is still alive > for MSVC builds; it just turns that into `--with-ssl=openssl` in the > fake CONFIGURE_ARGS? Yeah, I think that keeping both variables separated in the MSVC

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-01-27 Thread Joel Jacobson
Hi Mark, On Wed, Jan 27, 2021, at 22:36, Mark Rofail wrote: > Vectors as refrencing columns are not supported and out of scope of this > patch. Try to use arrays. OK, not a biggie, but I think the user at least should get an error message immediately when trying to add the foreign key on an

Re: logical replication worker accesses catalogs in error context callback

2021-01-27 Thread Amit Kapila
On Wed, Jan 27, 2021 at 9:38 AM Bharath Rupireddy wrote: > > On Wed, Jan 27, 2021 at 7:48 AM Zhihong Yu wrote: > > Thanks for pointing to the changes in the commit message. I corrected > them. Attaching v4 patch set, consider it for further review. > About 0001, have we tried to reproduce the

Re: On login trigger: take three

2021-01-27 Thread Amit Kapila
On Thu, Jan 28, 2021 at 8:17 AM Amit Kapila wrote: > > On Mon, Dec 28, 2020 at 5:46 PM Masahiko Sawada wrote: > > > > On Sat, Dec 26, 2020 at 4:04 PM Pavel Stehule > > wrote: > > > > > > > > > > > > so 26. 12. 2020 v 8:00 odesílatel Pavel Stehule > > > napsal: > > >> > > >> Hi > > >> > > >>

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread japin
On Thu, 28 Jan 2021 at 12:22, Bharath Rupireddy wrote: > On Wed, Jan 27, 2021 at 7:35 PM Li Japin wrote: >> > I don't see any problem if ALTER SUBSCRIPTION ... ADD PUBLICATION with >> > refresh true refreshes only the newly added publications, because what >> > we do in

Re: Is Recovery actually paused?

2021-01-27 Thread Dilip Kumar
On Wed, Jan 27, 2021 at 2:28 PM Dilip Kumar wrote: > > On Wed, Jan 27, 2021 at 2:06 PM Yugo NAGATA wrote: > > > > On Wed, 27 Jan 2021 13:29:23 +0530 > > Dilip Kumar wrote: > > > > > On Wed, Jan 27, 2021 at 12:50 PM Masahiko Sawada > > > wrote: > > > > > > > > On Tue, Jan 26, 2021 at 2:00 AM

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread Bharath Rupireddy
On Wed, Jan 27, 2021 at 7:35 PM Li Japin wrote: > > I don't see any problem if ALTER SUBSCRIPTION ... ADD PUBLICATION with > > refresh true refreshes only the newly added publications, because what > > we do in AlterSubscription_refresh() is that we fetch the tables > > associated with the

Re: Wrong usage of RelationNeedsWAL

2021-01-27 Thread Kyotaro Horiguchi
At Wed, 27 Jan 2021 02:48:48 -0800, Noah Misch wrote in > On Thu, Jan 21, 2021 at 01:23:36AM -0800, Noah Misch wrote: > > On Thu, Jan 21, 2021 at 06:02:11PM +0900, Kyotaro Horiguchi wrote: > > > Perhaps I'm missing something, but the patch doesn't pass the v5-0001 > > > test with

Re: On login trigger: take three

2021-01-27 Thread Amit Kapila
On Mon, Dec 28, 2020 at 5:46 PM Masahiko Sawada wrote: > > On Sat, Dec 26, 2020 at 4:04 PM Pavel Stehule wrote: > > > > > > > > so 26. 12. 2020 v 8:00 odesílatel Pavel Stehule > > napsal: > >> > >> Hi > >> > >> > >>> Thank you. > >>> I have applied all your fixes in

RE: [PoC] Non-volatile WAL buffer

2021-01-27 Thread tsunakawa.ta...@fujitsu.com
From: Tomas Vondra > (c) As mentioned before, PMEM behaves differently with concurrent > access, i.e. it reaches peak throughput with relatively low number of > threads wroting data, and then the throughput drops quite quickly. I'm > not sure if the same thing applies to pmem_drain() too - if it

RE: [POC] Fast COPY FROM command for the table with foreign partitions

2021-01-27 Thread Tang, Haiying
Hi Andrey, > Sometimes before i suggested additional optimization [1] which can > additionally speed up COPY by 2-4 times. Maybe you can perform the > benchmark for this solution too? Sorry for the late reply, I just have time to take this test now. But the patch no longer applies, I tried to

Re: row filtering for logical replication

2021-01-27 Thread Andres Freund
Hi, On 2020-12-17 09:43:30 +0300, Önder Kalacı wrote: > The above part can be considered the core of the logic, executed per tuple. > As far as I can see, it has two downsides. > > First, calling `expression_planner()` for every tuple can be quite > expensive. I created a sample table, loaded

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-01-27 Thread Peter Geoghegan
On Wed, Jan 27, 2021 at 5:24 PM Peter Geoghegan wrote: > The issue here is that it would also be nice to use a "recently dead" > bit on the primary, but if you do that then maybe you're back to the > original problem. OTOH, I also think that we could repurpose the > LP_NORMAL bit in index AMs, so

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-01-27 Thread Peter Geoghegan
On Wed, Jan 27, 2021 at 11:30 AM Michail Nikolaev wrote: > Sorry for necroposting, but if someone is interested - I hope the patch is > ready now and available in the other thread (1). I wonder if it would help to not actually use the LP_DEAD bit for this. Instead, you could use the

Re: Add MAIN_RELATION_CLEANUP and SECONDARY_RELATION_CLEANUP options to VACUUM

2021-01-27 Thread Justin Pryzby
On Wed, Jan 27, 2021 at 11:16:26PM +, Bossart, Nathan wrote: > On 1/27/21, 11:07 AM, "Justin Pryzby" wrote: > > This just came up for me: > > > > I have a daily maintenance script which pro-actively vacuums tables: > > freezing > > historic partitions, vacuuming current tables if the table's

Re: protect pg_stat_statements_info() for being used without the library loaded

2021-01-27 Thread Julien Rouhaud
On Thu, Jan 28, 2021 at 3:53 AM Jaime Casanova wrote: > > Hi, > > Attached is a small patch for ${subject} Good catch, and patch looks good to me.

RE: Enhance traceability of wal_level changes for backup management

2021-01-27 Thread osumi.takami...@fujitsu.com
Hello On Thursday, January 21, 2021 11:19 PM I wrote: > If no one disagree with it, I'll proceed with (1) below. > > (1) writing the time or LSN in the control file to indicate when/where > wal_level > is changed to 'minimal' > from upper level to invalidate the old backups or make alerts to

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-01-27 Thread Zhihong Yu
Hi, Mark: + if (ARR_NDIM(arr) != 1 || + ARR_HASNULL(arr) || + ARR_ELEMTYPE(arr) != CHAROID) + elog(ERROR, "confreftype is not a 1-D char array"); I think the ARR_HASNULL(arr) condition is not reflected in the error message. +* Array foreign keys support

Re: Perform COPY FROM encoding conversions in larger chunks

2021-01-27 Thread John Naylor
Hi Heikki, 0001 through 0003 are straightforward, and I think they can be committed now if you like. 0004 is also pretty straightforward. The check you proposed upthread for pg_upgrade seems like the best solution to make that workable. I'll take a look at 0005 soon. I measured the conversions

Re: vacuum_cost_page_miss default value and modern hardware

2021-01-27 Thread Peter Geoghegan
On Thu, Jan 21, 2021 at 5:12 PM Peter Geoghegan wrote: > I'm going to go ahead with committing my patch to lower the default > next week. If anybody has any objections to that plan, please speak > up. Just pushed a commit that reduced the default for vacuum_cost_page_miss to 2. One more thing

Re: Is it useful to record whether plans are generic or custom?

2021-01-27 Thread Tatsuro Yamada
Hi Toricoshi-san, On 2021/01/12 20:36, torikoshia wrote: I suppose it would be normal practice to store past results of pg_stat_statements for future comparisons. If this is the case, I think that if we only add the number of generic plan execution, it will give us a hint to notice the cause of

Re: Is it useful to record whether plans are generic or custom?

2021-01-27 Thread Tatsuro Yamada
Horiguchi-san, On 2020/12/04 15:37, Kyotaro Horiguchi wrote: And I'm also struggling with the following. | However, I also began to wonder how effective it would be to just | distinguish between generic and custom plans. Custom plans can | include all sorts of plans. and thinking cache

Re: Is it useful to record whether plans are generic or custom?

2021-01-27 Thread Tatsuro Yamada
Torikoshi-san, On 2020/12/04 15:03, torikoshia wrote: ISTM now that creating pg_stat_statements_xxx views both for generic andcustom plans is better than my PoC patch. And I'm also struggling with the following. | However, I also began to wonder how effective it would be to just |

Re: Is it useful to record whether plans are generic or custom?

2021-01-27 Thread Tatsuro Yamada
On 2020/12/04 14:29, Fujii Masao wrote: On 2020/11/30 15:24, Tatsuro Yamada wrote: Hi Torikoshi-san, In this patch, exposing new columns is mandatory, but I think it's better to make it optional by adding a GUC something like 'pgss.track_general_custom_plans. I also feel it makes the

Index predicate locking and serializability contention

2021-01-27 Thread Marcelo Zabani
Hi! We're currently having issues with serializable contention at our shop, and after tracking it down very carefully, we found that there are two main reasons for one such conflict: 1. Page-level predicate locks on primary key indexes, whose associated column gets their Id from a sequence. 2. An

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-27 Thread Alvaro Herrera
On 2021-Jan-28, Alexey Kondratov wrote: > I have read more about lock levels and ShareLock should prevent any kind of > physical modification of indexes. We already hold ShareLock doing > find_all_inheritors(), which is higher than ShareUpdateExclusiveLock, so > using ShareLock seems to be safe

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-01-27 Thread Mark Rofail
> > > Hey Joel, I apologize for my rash response, I did not quite understand your example > at first, it appears the UPDATE statement is neither over the > referencing nor the referenced columns > The v14 fixed the issue where an error would occur by an update statement that didn't target the

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-27 Thread Justin Pryzby
Thanks for updating the patch. I have just a couple comments on the new (and old) language. On Thu, Jan 28, 2021 at 12:19:06AM +0300, Alexey Kondratov wrote: > Also added tests for ACL checks, relfilenode changes. Added ACL recheck for > multi-transactional case. Added info about TOAST index

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-27 Thread Alexey Kondratov
On 2021-01-27 06:14, Michael Paquier wrote: On Wed, Jan 27, 2021 at 01:00:50AM +0300, Alexey Kondratov wrote: In the new 0002 I moved ACL check to the upper level, i.e. ExecReindex(), and removed expensive text generation in test. Not touched yet some of your previously raised concerns. Also,

Re: Allow matching whole DN from a client certificate

2021-01-27 Thread Andrew Dunstan
On 1/27/21 3:14 PM, Jacob Champion wrote: > On Tue, 2021-01-26 at 18:43 +, Jacob Champion wrote: >> On Tue, 2021-01-26 at 13:49 +0100, Daniel Gustafsson wrote: >>> The OpenSSL X509_NAME_cmp function use RFC 5280 section 7.1 and RFC 4517 >>> section 4.2.15 (which in turn reference RFC 4514

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-01-27 Thread Joel Jacobson
Hi Mark, On Wed, Jan 27, 2021, at 20:34, Mark Rofail wrote: >Here it is. >Array-ELEMENT-foreign-key-v15.patch Thanks. I've tested it and notice there still seems to be a problem with int2vector? Reading your previous comment a few messages ago, it sounds like this was fixed, but perhaps not?

Re: archive status ".ready" files may be created too early

2021-01-27 Thread Bossart, Nathan
On 1/26/21, 6:36 PM, "Kyotaro Horiguchi" wrote: > At Tue, 26 Jan 2021 19:13:57 +, "Bossart, Nathan" > wrote in >> On 12/17/20, 9:15 PM, "Kyotaro Horiguchi" wrote: >> > At Thu, 17 Dec 2020 22:20:35 +, "Bossart, Nathan" >> > wrote in >> >> On 12/15/20, 2:33 AM, "Kyotaro Horiguchi"

Re: Allow matching whole DN from a client certificate

2021-01-27 Thread Jacob Champion
On Tue, 2021-01-26 at 18:43 +, Jacob Champion wrote: > On Tue, 2021-01-26 at 13:49 +0100, Daniel Gustafsson wrote: > > The OpenSSL X509_NAME_cmp function use RFC 5280 section 7.1 and RFC 4517 > > section 4.2.15 (which in turn reference RFC 4514 for the DN string format). > > libnss has

protect pg_stat_statements_info() for being used without the library loaded

2021-01-27 Thread Jaime Casanova
Hi, Attached is a small patch for ${subject} -- Jaime Casanova Director de Servicios Profesionales SystemGuards - Consultores de PostgreSQL diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c index 72a117fc19..62cccbfa44 100644 ---

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-01-27 Thread Joel Jacobson
Hi Mark, On Wed, Jan 27, 2021, at 10:58, Mark Rofail wrote: >Kindly find the updated patch (v15) below I think you forgot to attach the patch. /Joel

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-01-27 Thread Michail Nikolaev
Hello, hackers. Sorry for necroposting, but if someone is interested - I hope the patch is ready now and available in the other thread (1). Thanks, Michail. [1] https://www.postgresql.org/message-id/flat/CANtu0oiP18H31dSaEzn0B0rW6tA_q1G7%3D9Y92%2BUS_WHGOoQevg%40mail.gmail.com

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2021-01-27 Thread Michail Nikolaev
Hello, hackers. I think I was able to fix the issue related to minRecoveryPoint and crash recovery. To make sure standby will be consistent after crash recovery, we need to take the current value of minRecoveryPoint into account while setting LP_DEAD hints (almost the same way as it is done for

Re: Add MAIN_RELATION_CLEANUP and SECONDARY_RELATION_CLEANUP options to VACUUM

2021-01-27 Thread Justin Pryzby
On Fri, Jan 24, 2020 at 09:24:45PM +, Bossart, Nathan wrote: > On 1/21/20, 1:39 PM, "Vik Fearing" wrote: > > On 21/01/2020 22:21, Bossart, Nathan wrote: > >> I've attached a patch for a couple of new options for VACUUM: > >> MAIN_RELATION_CLEANUP and SECONDARY_RELATION_CLEANUP. The motive >

Re: Support for NSS as a libpq TLS backend

2021-01-27 Thread Jacob Champion
On Wed, 2021-01-27 at 16:39 +0900, Michael Paquier wrote: > My apologies for chiming in. I was looking at your patch set here, > and while reviewing the strong random and cryptohash parts I have > found a couple of mistakes in the ./configure part. I think that the > switch from --with-openssl

Re: cleaning up a few CLOG-related things

2021-01-27 Thread Robert Haas
On Mon, Jan 25, 2021 at 2:11 PM Heikki Linnakangas wrote: > > [patches 0001 - 0003] > > Makes sense. Great. I committed the first one and will proceed with those as well. > > So, the CLOG page gets created when nextXid advances from the first > > value on the page to the second value on the

Re: Printing backtrace of postgres processes

2021-01-27 Thread Andres Freund
Hi, On 2021-01-27 19:05:16 +0530, vignesh C wrote: > /* > + * LogBackTrace > + * > + * Get the backtrace and log the backtrace to log file. > + */ > +void > +LogBackTrace(void) > +{ > + int save_errno = errno; > + > + void *buf[100]; > + int

Improve join selectivity estimation using extended statistics

2021-01-27 Thread Konstantin Knizhnik
Original thread is: https://www.postgresql.org/message-id/flat/196f1e1a-5464-ed07-ab3c-0c9920564af7%40postgrespro.ru Following Yugo's advice, I have splitted this patch into two: 1. Extending auto_explain extension to generate extended statistics in case of bad selectivity estimation. 2. Taken

Re: [PoC] Non-volatile WAL buffer

2021-01-27 Thread Tomas Vondra
On 1/25/21 3:56 AM, Masahiko Sawada wrote: >> >> ... >> >> On 1/21/21 3:17 AM, Masahiko Sawada wrote: >>> ... >>> >>> While looking at the two methods: NTT and simple-no-buffer, I realized >>> that in XLogFlush(), NTT patch flushes (by pmem_flush() and >>> pmem_drain()) WAL without acquiring

Re: Columns correlation and adaptive query optimization

2021-01-27 Thread Konstantin Knizhnik
On 27.01.2021 8:45, Yugo NAGATA wrote: On Mon, 25 Jan 2021 16:27:25 +0300 Konstantin Knizhnik wrote: Hello, Thank you for review. My answers are inside. Thank you for updating the patch and answering my questions. (2) If I understand correctly, your proposal consists of the following

Inconsistent function definitions in ECPG's informix.c

2021-01-27 Thread Tom Lane
I noticed that some of the newer compilers in the buildfarm (e.g., caiman, with gcc 11.0) whine about the definitions of rjulmdy() and rmdyjul() not quite matching their external declarations: informix.c:516:23: warning: argument 2 of type `short int[3]' with mismatched bound

Re: Online checksums patch - once again

2021-01-27 Thread Heikki Linnakangas
Revisiting an issue we discussed earlier: On 25/11/2020 15:20, Daniel Gustafsson wrote: On 23 Nov 2020, at 18:36, Heikki Linnakangas wrote: On 17/11/2020 10:56, Daniel Gustafsson wrote: I've reworked this in the attached such that the enable_ and disable_ functions merely call into the

Re: Speeding up GIST index creation for tsvectors

2021-01-27 Thread Amit Khandekar
On Tue, 15 Dec 2020 at 20:34, Amit Khandekar wrote: > > On Sun, 13 Dec 2020 at 9:28 PM, Andrey Borodin wrote: > > +1 > > This will make all INSERTs and UPDATES for tsvector's GiSTs. > > Oh, I didn't realize that this code is getting used in GIST index > insertion and creation too. Will check

Re: Key management with tests

2021-01-27 Thread Bruce Momjian
On Tue, Jan 26, 2021 at 05:53:01PM -0500, Bruce Momjian wrote: > On Tue, Jan 26, 2021 at 03:24:30PM -0500, Robert Haas wrote: > > I'm wondering whether you've considered storing all the keys in one > > file instead of a file per key. The reason I ask is that it seems to > > me that the key

Re: FailedAssertion in heap_index_delete_tuples at heapam.c:7220

2021-01-27 Thread Jaime Casanova
On Wed, Jan 27, 2021 at 2:09 AM Peter Geoghegan wrote: > > On Tue, Jan 26, 2021 at 10:52 PM Jaime Casanova > wrote: > > ${subject} happened while executing ${attached query} at regresssion > > database, using 14dev (commit > > d5a83d79c9f9b660a6a5a77afafe146d3c8c6f46) and produced ${attached > >

Re: [PATCH] Disable bgworkers during servers start in pg_upgrade

2021-01-27 Thread Jehan-Guillaume de Rorthais
Oh, I forgot another point before sending my previous email. Maybe it might worth adding some final safety checks in pg_upgrade itself? Eg. checking controldata and mxid files coherency between old and new cluster would have catch the inconsistency here.

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread Li Japin
> On Jan 27, 2021, at 19:41, Bharath Rupireddy > wrote: > > On Wed, Jan 27, 2021 at 4:42 PM Amit Kapila wrote: >>> On Wed, Jan 27, 2021 at 3:16 PM Bharath Rupireddy >>> wrote: On Wed, Jan 27, 2021 at 3:01 PM Amit Kapila wrote: >>> So, I think the new syntax, ALTER SUBSCRIPTION

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread Li Japin
> On Jan 27, 2021, at 19:41, Bharath Rupireddy > wrote: > > On Wed, Jan 27, 2021 at 4:42 PM Amit Kapila wrote: >>> On Wed, Jan 27, 2021 at 3:16 PM Bharath Rupireddy >>> wrote: On Wed, Jan 27, 2021 at 3:01 PM Amit Kapila wrote: >>> So, I think the new syntax, ALTER SUBSCRIPTION

Re: [PATCH] Disable bgworkers during servers start in pg_upgrade

2021-01-27 Thread Jehan-Guillaume de Rorthais
Hi, On Wed, 27 Jan 2021 11:25:11 +0100 Denis Laxalde wrote: > Andres Freund a écrit : > > On 2021-01-21 16:23:58 +0100, Denis Laxalde wrote: > > > We found an issue in pg_upgrade on a cluster with a third-party > > > background worker. The upgrade goes fine, but the new cluster is then in >

Re: Printing backtrace of postgres processes

2021-01-27 Thread vignesh C
On Thu, Jan 21, 2021 at 7:26 AM Tom Lane wrote: > > Craig Ringer writes: > > On Wed, 20 Jan 2021 at 05:23, Tom Lane wrote: > >> BTW, it also looks like the patch is doing nothing to prevent the > >> backtrace from being sent to the connected client. > > > I don't see a good reason to send a bt

Re: Fix ALTER SUBSCRIPTION ... SET PUBLICATION documentation

2021-01-27 Thread japin
On Wed, 27 Jan 2021 at 19:47, Bharath Rupireddy wrote: > On Wed, Jan 27, 2021 at 4:57 PM Amit Kapila wrote: >> >> On Tue, Jan 26, 2021 at 4:56 PM japin wrote: >> > >> > >> > Hi, >> > >> > When I read the documentation of ALTER SUBSCRIPTION ... SET PUBLICATION >> > ... WITH (...), >> > it

Re: simplifying foreign key/RI checks

2021-01-27 Thread Amit Langote
On Wed, Jan 27, 2021 at 5:32 PM Kyotaro Horiguchi wrote: > At Sun, 24 Jan 2021 20:51:39 +0900, Amit Langote > wrote in > > Here's v5. > > At Mon, 25 Jan 2021 18:19:56 +0900, Amit Langote > wrote in > > > Anybody else want to look this patch over before I mark it Ready For > > > Committer? >

Re: Fix ALTER SUBSCRIPTION ... SET PUBLICATION documentation

2021-01-27 Thread Bharath Rupireddy
On Wed, Jan 27, 2021 at 4:57 PM Amit Kapila wrote: > > On Tue, Jan 26, 2021 at 4:56 PM japin wrote: > > > > > > Hi, > > > > When I read the documentation of ALTER SUBSCRIPTION ... SET PUBLICATION ... > > WITH (...), > > it says "set_publication_option" only support "refresh" in documentation >

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread Bharath Rupireddy
On Wed, Jan 27, 2021 at 4:42 PM Amit Kapila wrote: > > On Wed, Jan 27, 2021 at 3:16 PM Bharath Rupireddy > wrote: > > > > On Wed, Jan 27, 2021 at 3:01 PM Amit Kapila wrote: > > > > So, I think the new syntax, ALTER SUBSCRIPTION .. ADD/DROP PUBLICATION > > will refresh the new and existing

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-27 Thread Greg Nancarrow
On Wed, Jan 27, 2021 at 2:13 PM Hou, Zhijie wrote: > > Hi, > > When testing the patch with the following kind of sql. > > --- > Insert into part_table select 1; > Insert into part_table select generate_series(1,1,1); > Insert into part_table select * from testfunc(); > --- > > we usually use

Re: Fix ALTER SUBSCRIPTION ... SET PUBLICATION documentation

2021-01-27 Thread Amit Kapila
On Tue, Jan 26, 2021 at 4:56 PM japin wrote: > > > Hi, > > When I read the documentation of ALTER SUBSCRIPTION ... SET PUBLICATION ... > WITH (...), > it says "set_publication_option" only support "refresh" in documentation [1]. > However, we can also supply the "copy_data" option, and the code

Re: pg_upgrade fails with non-standard ACL

2021-01-27 Thread Noah Misch
On Mon, Jan 25, 2021 at 10:14:43PM +0300, Anastasia Lubennikova wrote: > On 24.01.2021 11:39, Noah Misch wrote: > >On Thu, Jan 21, 2021 at 01:03:58AM +0300, Anastasia Lubennikova wrote: > >>On 03.01.2021 14:29, Noah Misch wrote: > >>>Overall, this patch predicts a subset of cases where pg_dump

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread Amit Kapila
On Wed, Jan 27, 2021 at 3:16 PM Bharath Rupireddy wrote: > > On Wed, Jan 27, 2021 at 3:01 PM Amit Kapila wrote: > > So, I think the new syntax, ALTER SUBSCRIPTION .. ADD/DROP PUBLICATION > will refresh the new and existing publications. > That sounds a bit unusual to me because when the user

Re: Protect syscache from bloating with negative cache entries

2021-01-27 Thread Heikki Linnakangas
On 27/01/2021 03:13, Kyotaro Horiguchi wrote: At Thu, 14 Jan 2021 17:32:27 +0900 (JST), Kyotaro Horiguchi wrote in The commit 4656e3d668 (debug_invalidate_system_caches_always) conflicted with this patch. Rebased. At Wed, 27 Jan 2021 10:07:47 +0900 (JST), Kyotaro Horiguchi wrote in (I

Re: Wrong usage of RelationNeedsWAL

2021-01-27 Thread Noah Misch
On Thu, Jan 21, 2021 at 01:23:36AM -0800, Noah Misch wrote: > On Thu, Jan 21, 2021 at 06:02:11PM +0900, Kyotaro Horiguchi wrote: > > At Thu, 21 Jan 2021 00:19:58 -0800, Noah Misch wrote in > > > On Thu, Jan 21, 2021 at 12:28:44AM +0900, Kyotaro Horiguchi wrote: > > > > However, with the previous

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread Dilip Kumar
On Wed, Jan 27, 2021 at 3:26 PM japin wrote: > > > On Wed, 27 Jan 2021 at 16:59, Amit Kapila wrote: > > On Tue, Jan 26, 2021 at 9:18 AM japin wrote: > >> > >> > >> When I read the discussion in [1], I found that update subscription's > >> publications > >> is complicated. > >> > >> For

Re: [PATCH] Disable bgworkers during servers start in pg_upgrade

2021-01-27 Thread Denis Laxalde
Hi, Andres Freund a écrit : > On 2021-01-21 16:23:58 +0100, Denis Laxalde wrote: > > We found an issue in pg_upgrade on a cluster with a third-party > > background worker. The upgrade goes fine, but the new cluster is then in > > an inconsistent state. The background worker comes from the PoWA >

Re: poc - possibility to write window function in PL languages

2021-01-27 Thread Pavel Stehule
st 20. 1. 2021 v 21:14 odesílatel Pavel Stehule napsal: > > > st 20. 1. 2021 v 21:07 odesílatel Tom Lane napsal: > >> Pavel Stehule writes: >> > The second question is work with partition context value. This should be >> > only one value, and of only one but of any type per function. In this

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-01-27 Thread Mark Rofail
Hi Joel, As always, great catch! Kindly find the updated patch (v15) below Changelog: - v15 (compatible with current master 2021-01-27, commit e42b3c3bd6a9c6233ac4c8a2e9b040367ba2f97c) * remove "EACH ELEMENT OF" from violation messages * accommodate tests accordingly Keep up the good

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread japin
On Wed, 27 Jan 2021 at 16:59, Amit Kapila wrote: > On Tue, Jan 26, 2021 at 9:18 AM japin wrote: >> >> >> When I read the discussion in [1], I found that update subscription's >> publications >> is complicated. >> >> For example, I have 5 publications in subscription. >> >> CREATE

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread japin
On Wed, 27 Jan 2021 at 17:46, Bharath Rupireddy wrote: > On Wed, Jan 27, 2021 at 3:01 PM Amit Kapila wrote: >> > > While the new proposed syntax does seem to provide some ease for users >> > > but it has nothing which we can't do with current syntax. Also, in the >> > > current syntax, there

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread Bharath Rupireddy
On Wed, Jan 27, 2021 at 3:01 PM Amit Kapila wrote: > > > While the new proposed syntax does seem to provide some ease for users > > > but it has nothing which we can't do with current syntax. Also, in the > > > current syntax, there is an additional provision for refreshing the > > > existing

Re: Freeze the inserted tuples during CTAS?

2021-01-27 Thread Komяpa
Hi, I confirm that my analytic workflows often do the CTAS and VACUUM of the relation right after, before the index creation, to mark stuff as all-visible for IOS to work. Freezing and marking as visible will help. On Wed, Jan 27, 2021 at 12:29 PM Paul Guo wrote: > Here is the simple patch, >

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread Amit Kapila
On Wed, Jan 27, 2021 at 2:57 PM Bharath Rupireddy wrote: > > On Wed, Jan 27, 2021 at 2:30 PM Amit Kapila wrote: > > > > On Tue, Jan 26, 2021 at 9:18 AM japin wrote: > > > > > > > > > When I read the discussion in [1], I found that update subscription's > > > publications > > > is complicated.

Freeze the inserted tuples during CTAS?

2021-01-27 Thread Paul Guo
Here is the simple patch, diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c index dce882012e..0391699423 100644 --- a/src/backend/commands/createas.c +++ b/src/backend/commands/createas.c @@ -552,7 +552,7 @@ intorel_startup(DestReceiver *self, int operation,

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread Bharath Rupireddy
On Wed, Jan 27, 2021 at 2:30 PM Amit Kapila wrote: > > On Tue, Jan 26, 2021 at 9:18 AM japin wrote: > > > > > > When I read the discussion in [1], I found that update subscription's > > publications > > is complicated. > > > > For example, I have 5 publications in subscription. > > > >

Two patches to speed up pg_rewind.

2021-01-27 Thread Paul Guo
While reading pg_rewind code I found two things could speed up pg_rewind. Attached are the patches. First one: pg_rewind would fsync the whole pgdata directory on the target by default, but that is a waste since usually just part of the files/directories on the target are modified. Other files

RE: ECPG: proposal for new DECLARE STATEMENT

2021-01-27 Thread kuroda.hay...@fujitsu.com
Dear Hackers, I know I'm asking a big favor, but could you review(and commit) the patch? The status has become RFC last Nov., but no one checked this after that. Maybe Meskes is quite busy and have no time to review it. The main part of the patch is about 200 lines(It means not so long), and

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread Amit Kapila
On Tue, Jan 26, 2021 at 9:18 AM japin wrote: > > > When I read the discussion in [1], I found that update subscription's > publications > is complicated. > > For example, I have 5 publications in subscription. > > CREATE SUBSCRIPTION mysub1 CONNECTION 'host=localhost port=5432 >

Re: Is Recovery actually paused?

2021-01-27 Thread Dilip Kumar
On Wed, Jan 27, 2021 at 2:06 PM Yugo NAGATA wrote: > > On Wed, 27 Jan 2021 13:29:23 +0530 > Dilip Kumar wrote: > > > On Wed, Jan 27, 2021 at 12:50 PM Masahiko Sawada > > wrote: > > > > > > On Tue, Jan 26, 2021 at 2:00 AM Robert Haas wrote: > > > > > > > > On Sat, Jan 23, 2021 at 6:10 AM

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2021-01-27 Thread Paul Guo
Thanks for the review, please see the replies below. > On Jan 5, 2021, at 9:07 AM, Kyotaro Horiguchi wrote: > > At Wed, 8 Jul 2020 12:56:44 +, Paul Guo wrote in >> On 2020/01/15 19:18, Paul Guo wrote: >> I further fixed the last test failure (due to a small bug in the test, not >> in

Re: Is Recovery actually paused?

2021-01-27 Thread Yugo NAGATA
On Wed, 27 Jan 2021 13:29:23 +0530 Dilip Kumar wrote: > On Wed, Jan 27, 2021 at 12:50 PM Masahiko Sawada > wrote: > > > > On Tue, Jan 26, 2021 at 2:00 AM Robert Haas wrote: > > > > > > On Sat, Jan 23, 2021 at 6:10 AM Bharath Rupireddy > > > wrote: > > > > +1 to just show the recovery pause

Re: simplifying foreign key/RI checks

2021-01-27 Thread Kyotaro Horiguchi
At Sun, 24 Jan 2021 20:51:39 +0900, Amit Langote wrote in > Here's v5. At Mon, 25 Jan 2021 18:19:56 +0900, Amit Langote wrote in > > Anybody else want to look this patch over before I mark it Ready For > > Committer? > > Would be nice to have others look it over. Thanks. This nice

Re: [PoC] Non-volatile WAL buffer

2021-01-27 Thread Takashi Menjo
Hi, Now I have caught up with this thread. I see that many of you are interested in performance profiling. I share my slides in SNIA SDC 2020 [1]. In the slides, I had profiles focused on XLogInsert and XLogFlush (mainly the latter) for my non-volatile WAL buffer patchset. I found that the time

Re: Parallel Inserts in CREATE TABLE AS

2021-01-27 Thread Bharath Rupireddy
On Wed, Jan 27, 2021 at 1:25 PM Tang, Haiying wrote: > I choose 5 cases which pick parallel insert plan in CTAS to measure the > patched performance. Each case run 30 times. > > Most of the tests execution become faster with this patch. > > However, Test NO 4(create table xxx as table xxx.)

Re: [PATCH] remove pg_standby

2021-01-27 Thread Fujii Masao
On 2021/01/27 14:32, Thomas Munro wrote: On Wed, Jan 27, 2021 at 6:06 PM Michael Paquier wrote: On Wed, Jan 27, 2021 at 04:13:24PM +1300, Thomas Munro wrote: I would like to commit this, because "waiting restore commands" have confusing interactions with my proposed

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread Dilip Kumar
On Tue, Jan 26, 2021 at 9:18 AM japin wrote: > > > Hi, hackers > > When I read the discussion in [1], I found that update subscription's > publications > is complicated. > > For example, I have 5 publications in subscription. > > CREATE SUBSCRIPTION mysub1 CONNECTION 'host=localhost