Re: .ready and .done files considered harmful

2021-07-19 Thread Dilip Kumar
On Mon, Jul 19, 2021 at 5:43 PM Dipesh Pandit wrote: > > Hi, > > > I agree, I missed this part. The .history file should be given higher > > preference. > > I will take care of it in the next patch. > > Archiver does not have access to shared memory and the current timeline ID > is not available

Re: row filtering for logical replication

2021-07-19 Thread Amit Kapila
On Mon, Jul 19, 2021 at 7:02 PM Tomas Vondra wrote: > > On 7/19/21 1:00 PM, Dilip Kumar wrote: > > On Mon, Jul 19, 2021 at 3:12 PM Amit Kapila wrote: > >> a. Just log it and move to the next row > >> b. send to stats collector some info about this which can be displayed > >> in a view and then

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2021-07-19 Thread David Rowley
On Mon, 12 Jul 2021 at 19:23, David Rowley wrote: > I also adjusted the hash seq scan code so that it performs better when > faced a non-sparsely populated table. Previously my benchmark for > that case didn't do well [2]. I was running some select only pgbench tests today on an AMD 3990x

Re: O_DIRECT on macOS

2021-07-19 Thread Thomas Munro
On Tue, Jul 20, 2021 at 12:26 PM Tom Lane wrote: > > I can try that on the gcc farm in a bit. Thanks! > Hmm, it compiles cleanly, but something seems drastically wrong, > because performance is just awful. On the other hand, I don't > know what sort of storage is underlying this instance, so

Re: Avoid stack frame setup in performance critical routines using tail calls

2021-07-19 Thread David Rowley
On Tue, 20 Jul 2021 at 08:00, Andres Freund wrote: > I have *not* carefully benchmarked this, but a quick implementation of this > does seem to increase readonly pgbench tps at a small scale by 2-3% (both Interesting. I've not taken the time to study the patch but I was running some other

Re: Introduce pg_receivewal gzip compression tests

2021-07-19 Thread Michael Paquier
On Mon, Jul 19, 2021 at 04:03:33PM +0900, Michael Paquier wrote: > Another advantage of this patch is the handling of ".gz" is reduced to > one code path instead of four. That makes a bit easier the > introduction of new compression methods. > > A second thing that was really confusing is that

Re: row filtering for logical replication

2021-07-19 Thread Amit Kapila
On Mon, Jul 19, 2021 at 4:31 PM Dilip Kumar wrote: > > On Mon, Jul 19, 2021 at 3:12 PM Amit Kapila wrote: > > > > Maybe a second option is to have replication change any UPDATE into > > > either an INSERT or a DELETE, if the old or the new row do not pass the > > > filter, respectively. That

Re: Have I found an interval arithmetic bug?

2021-07-19 Thread Bruce Momjian
On Wed, Jul 14, 2021 at 09:03:21AM -0700, Zhihong Yu wrote: > On Thu, Jul 8, 2021 at 10:22 AM Zhihong Yu wrote: > On Wed, Jun 30, 2021 at 9:35 AM Bruce Momjian wrote: > > On Tue, Jun 29, 2021 at 06:49:45PM +0200, Daniel Gustafsson wrote: > > > On 29 Jun 2021, at 18:50,

Re: Failure with 004_logrotate in prairiedog

2021-07-19 Thread Kyotaro Horiguchi
At Mon, 19 Jul 2021 10:23:46 -0400, Tom Lane wrote in > Michael Paquier writes: > > On Mon, Jul 19, 2021 at 04:15:36PM +0900, Kyotaro Horiguchi wrote: > >> When rotation happens, the metainfo file is once removed then > >> created. If slurp_file in the metafile-checking loop hits the gap, the >

Re: [HACKERS] logical decoding of two-phase transactions

2021-07-19 Thread Peter Smith
On Fri, Jul 16, 2021 at 4:08 PM vignesh C wrote: > [...] > Thanks for the updated patch, the patch applies cleanly and test passes: > I had couple of comments: > 1) Should we include "stream_prepare_cb" here in > logicaldecoding-streaming section of logicaldecoding.sgml > documentation: > To

Re: [HACKERS] logical decoding of two-phase transactions

2021-07-19 Thread Peter Smith
On Mon, Jul 19, 2021 at 3:28 PM Greg Nancarrow wrote: > > On Wed, Jul 14, 2021 at 6:33 PM Peter Smith wrote: > > > > Please find attached the latest patch set v97* > > > > I couldn't spot spot any significant issues in the v97-0001 patch, but > do have the following trivial feedback comments: >

Re: [HACKERS] logical decoding of two-phase transactions

2021-07-19 Thread Peter Smith
Please find attached the latest patch set v98* Patches: v97-0001 --> v98-0001 Differences: * Rebased to HEAD @ yesterday. * Code/Docs changes: 1. Fixed the same strcpy problem as reported by Tom Lane [1] for the previous 2PC patch. 2. Addressed all feedback suggestions given by Greg [2].

Kerberos delegation support in libpq and postgres_fdw

2021-07-19 Thread Peifeng Qiu
Hi hackers. This is the patch to add kerberos delegation support in libpq, which enables postgres_fdw to connect to another server and authenticate as the same user to the current login user. This will obsolete my previous patch which requires keytab file to be present on the fdw server host.

Re: Parallel INSERT SELECT take 2

2021-07-19 Thread Greg Nancarrow
On Tue, Jul 20, 2021 at 11:47 AM houzj.f...@fujitsu.com wrote: > > Attach rebased patches. > Just letting you know that CRLFs are in the patch comments for the 0001 and 0003 patches. (It doesn't affect patch application) Regards, Greg Nancarrow Fujitsu Australia

Re: Transactions and indexes

2021-07-19 Thread Peter Geoghegan
On Mon, Jul 19, 2021 at 7:20 PM Chris Cleveland wrote: > Thank you. Does this mean I can implement the index AM and return TIDs > without having to worry about transactions at all? Yes. That's the upside of the design -- it makes it easy to add new transactional index AMs. Which is one reason

Re: row filtering for logical replication

2021-07-19 Thread Alvaro Herrera
On 2021-Jul-19, Tomas Vondra wrote: > I have a feeling it's getting overly complicated, to the extent that > it'll be hard to explain to users and reason about. I don't think > there's a "perfect" solution for cases when the filter expression gives > different answers for old/new row - it'll

Re: Why ALTER SUBSCRIPTION ... SET (slot_name='none') requires subscription disabled?

2021-07-19 Thread Japin Li
On Mon, 19 Jul 2021 at 17:02, Amit Kapila wrote: > On Fri, Jul 16, 2021 at 2:12 PM Japin Li wrote: >> >> >> On Fri, 16 Jul 2021 at 14:06, Amit Kapila wrote: >> > On Fri, Jul 9, 2021 at 8:20 AM Japin Li wrote: >> >> >> >> On Thu, 08 Jul 2021 at 18:17, Amit Kapila wrote: >> >> > On Thu, Jul

Re: Rename of triggers for partitioned tables

2021-07-19 Thread Alvaro Herrera
On 2021-Jul-20, Arne Roland wrote: > Is your patch based on master? It doesn't apply at my end. It does ... master being dd498998a3 here, $ patch -p1 < /tmp/renametrig-8.patch patching file src/backend/commands/trigger.c patching file src/backend/parser/gram.y patching file

Re: Skipping logical replication transactions on subscriber side

2021-07-19 Thread Masahiko Sawada
On Mon, Jul 19, 2021 at 8:38 PM houzj.f...@fujitsu.com wrote: > > On July 19, 2021 2:40 PM Masahiko Sawada wrote: > > I've attached the updated version patch that incorporated all comments > > I got so far except for the clearing error details part I mentioned > > above. After getting a

Re: Skipping logical replication transactions on subscriber side

2021-07-19 Thread Masahiko Sawada
On Mon, Jul 19, 2021 at 5:47 PM Amit Kapila wrote: > > On Mon, Jul 19, 2021 at 12:10 PM Masahiko Sawada > wrote: > > > > On Sat, Jul 17, 2021 at 12:02 AM Masahiko Sawada > > wrote: > > > > > > 1. How to clear apply worker errors. IIUC we've discussed that once > > > the apply worker skipped

Re: Rename of triggers for partitioned tables

2021-07-19 Thread Arne Roland
> We are referring the trigger via it's name after all. If a child is named > differently we break with that assumption. I think informing the user about > that, is very valuable. To elaborate on that: While we to similar things for things like set schema, here it has a functional relevance.

RE: row filtering for logical replication

2021-07-19 Thread houzj.f...@fujitsu.com
Hi, I am interested in this feature and took a quick a look at the patch. Here are a few comments. (1) + appendStringInfo(, "%s", q); We'd better use appendStringInfoString(, q); (2) + whereclause = transformWhereClause(pstate, +

Re: O_DIRECT on macOS

2021-07-19 Thread Tom Lane
I wrote: > Thomas Munro writes: >> While I was here again, I couldn't resist trying to extend this to >> Solaris, since it looked so easy. I don't have access, but I tested >> on Illumos by undefining O_DIRECT. Thoughts? > I can try that on the gcc farm in a bit. Hmm, it compiles cleanly, but

Re: Rename of triggers for partitioned tables

2021-07-19 Thread Arne Roland
Hi! From: Alvaro Herrera Sent: Tuesday, July 20, 2021 02:03 To: Arne Roland Cc: vignesh C; Zhihong Yu; Pg Hackers Subject: Re: Rename of triggers for partitioned tables I found this coding too convoluted, so I rewrote it in a different way. You tests pass with this, but I admit I haven't

Re: row filtering for logical replication

2021-07-19 Thread Greg Nancarrow
On Mon, Jul 19, 2021 at 11:32 PM Tomas Vondra wrote: > > I have a feeling it's getting overly complicated, to the extent that > it'll be hard to explain to users and reason about. I don't think > there's a "perfect" solution for cases when the filter expression gives > different answers for

Re: Rename of triggers for partitioned tables

2021-07-19 Thread Alvaro Herrera
I found this coding too convoluted, so I rewrote it in a different way. You tests pass with this, but I admit I haven't double-checked them yet; I'll do that next. I don't think we need to give a NOTICE when the trigger name does not match; it doesn't really matter that the trigger was named

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-19 Thread Andres Freund
Hi, On 2021-07-19 16:49:15 -0700, Andres Freund wrote: > E.g. for > > select prepare( > 100, -- max block > 20, -- # of dead tuples per page > 10, -- dead tuples interval within a page > 1 -- page inteval > ); > attach sizeshuffled ordered > array69 ms 120 MB 84.87 s

Re: Transactions and indexes

2021-07-19 Thread Peter Geoghegan
On Mon, Jul 19, 2021 at 4:31 PM Chris Cleveland wrote: > I'm confused on how to handle transactions and visibility. In Postgres, indexes are not considered to be part of the logical database. They're just data structures that point to TIDs in the table. To an index, each TID is just another

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-19 Thread Andres Freund
Hi, On 2021-07-19 15:20:54 +0900, Masahiko Sawada wrote: > BTW is the implementation of the radix tree approach available > somewhere? If so I'd like to experiment with that too. > > > > > I have toyed with implementing adaptively large radix nodes like > > proposed in

Re: O_DIRECT on macOS

2021-07-19 Thread Tom Lane
Thomas Munro writes: > While I was here again, I couldn't resist trying to extend this to > Solaris, since it looked so easy. I don't have access, but I tested > on Illumos by undefining O_DIRECT. Thoughts? I can try that on the gcc farm in a bit. regards, tom lane

Transactions and indexes

2021-07-19 Thread Chris Cleveland
Noob here. I'm trying to implement a new type of index access method. I don't think I can use the built-in storage manager and buffer manager efficiently because files with 8k blocks aren't going to work. I really need to implement a log-structured file. I'm confused on how to handle

Re: OpenSSL 3.0.0 compatibility

2021-07-19 Thread Daniel Gustafsson
> On 3 Jul 2021, at 17:00, Peter Eisentraut > wrote: > > On 12.03.21 08:51, Peter Eisentraut wrote: >> On 11.03.21 11:41, Daniel Gustafsson wrote: >>> .. and apply the padding changes as proposed in a patch upthread like this >>> (these >>> work for all OpenSSL versions I've tested, and I'm

Re: O_DIRECT on macOS

2021-07-19 Thread Thomas Munro
On Tue, Jul 20, 2021 at 2:13 AM Tom Lane wrote: > Hmm ... we used to have to avoid putting #if constructs in the arguments > of macros (such as StaticAssertStmt). Maybe that's not a thing anymore > with C99, and in any case this whole stanza is fairly platform-specific > so we may not run into a

Micro-optimizations to avoid some strlen calls.

2021-07-19 Thread Ranier Vilela
Hi, There are some places, where strlen can have an overhead. This patch tries to fix this. Pass check-world at linux ubuntu (20.04) 64 bits. regards, Ranier Vilela diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index c14ca27c5e..2036072990 100644 ---

Re: slab allocator performance issues

2021-07-19 Thread Ranier Vilela
Em seg., 19 de jul. de 2021 às 17:56, Andres Freund escreveu: > Hi, > > On 2021-07-18 19:23:31 +0200, Tomas Vondra wrote: > > Sounds great! Thanks for investigating this and for the improvements. > > > > It might be good to do some experiments to see how the changes affect > memory > >

Re: slab allocator performance issues

2021-07-19 Thread Andres Freund
Hi, On 2021-07-18 19:23:31 +0200, Tomas Vondra wrote: > Sounds great! Thanks for investigating this and for the improvements. > > It might be good to do some experiments to see how the changes affect memory > consumption for practical workloads. I'm willing to spend soem time on that, > if

Avoid stack frame setup in performance critical routines using tail calls

2021-07-19 Thread Andres Freund
Hi, We have a few routines that are taking up a meaningful share of nearly all workloads. They are worth micro-optimizing, even though they rarely the most expensive parts of a profile. The memory allocation infrastructure is an example of that. When looking at a profile one can often see that a

Re: Support for NSS as a libpq TLS backend

2021-07-19 Thread Jacob Champion
On Wed, 2021-06-23 at 15:48 +0200, Daniel Gustafsson wrote: > Attached is a rebased version which incorporates your recent patchset for > resource handling, as well as the connect_ok test patch. With v38 I do see the "one-time function was previously called and failed" message you mentioned

Re: refactoring basebackup.c

2021-07-19 Thread Mark Dilger
> On Jul 8, 2021, at 8:56 AM, Robert Haas wrote: > > The interesting > patches in terms of functionality are 0006 and 0007; The difficulty in v3-0007 with pg_basebackup only knowing how to parse tar archives seems to be a natural consequence of not sufficiently abstracting out the handling

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2021-07-19 Thread Fabien COELHO
I attached the updated patch. # About pgbench error handling v15 Patches apply cleanly. Compilation, global and local tests ok. - v15.1: refactoring is a definite improvement. Good, even if it is not very useful (see below). While restructuring, maybe predefined variables could be

Re: Added documentation for cascade and restrict option of drop statistics

2021-07-19 Thread vignesh C
On Mon, 19 Jul 2021, 09:16 Michael Paquier, wrote: > > On Sun, Jul 18, 2021 at 12:37:48PM +0900, Michael Paquier wrote: > > Indeed, good catch. The other commands document that, so let's fix > > it. > > Applied. Thanks for committing this patch. Regards, Vignesh

Re: speed up verifying UTF-8

2021-07-19 Thread John Naylor
On Mon, Jul 19, 2021 at 9:43 AM Vladimir Sitnikov < sitnikov.vladi...@gmail.com> wrote: > It looks like it is important to have shrx for x86 which appears only when -march=x86-64-v3 is used (see https://github.com/golang/go/issues/47120#issuecomment-877629712 ). > Just in case: I know x86 wound

Re: refactoring basebackup.c

2021-07-19 Thread Dilip Kumar
On Mon, Jul 19, 2021 at 6:02 PM tushar wrote: > > On 7/16/21 12:43 PM, Dilip Kumar wrote: > > I think the problem is that bbsink_gzip_end_archive() is not > > forwarding the end request to the next bbsink. The attached patch so > > fix it. > > Thanks Dilip. Reported issue seems to be fixed now

Re: postgresql.conf.sample missing units

2021-07-19 Thread John Naylor
On Mon, Jul 19, 2021 at 5:44 AM Pavel Luzanov wrote: > > Hello, > > I found that the start section of the postgresql.conf file is missing a > description of two units: bytes (appeared in version 11) and > microseconds (in version 12). > > The attached patch makes these changes to the

Re: Failure with 004_logrotate in prairiedog

2021-07-19 Thread Tom Lane
Michael Paquier writes: > On Mon, Jul 19, 2021 at 04:15:36PM +0900, Kyotaro Horiguchi wrote: >> When rotation happens, the metainfo file is once removed then >> created. If slurp_file in the metafile-checking loop hits the gap, the >> slurp_file fails with ENOENT. > I can read the following

Re: Failure with 004_logrotate in prairiedog

2021-07-19 Thread Tom Lane
Kyotaro Horiguchi writes: > When rotation happens, the metainfo file is once removed then > created. If slurp_file in the metafile-checking loop hits the gap, the > slurp_file fails with ENOENT. Oh! Yeah, that's dumb, we should fix it to use rename(). Can't blame platform's rename() if it's

Re: O_DIRECT on macOS

2021-07-19 Thread Tom Lane
Thomas Munro writes: > On Mon, Jul 19, 2021 at 4:42 PM Tom Lane wrote: >> prairiedog thinks that Assert is too optimistic about whether all >> those flags exist. > Fixed. Hmm ... we used to have to avoid putting #if constructs in the arguments of macros (such as StaticAssertStmt). Maybe

Re: Question about non-blocking mode in libpq

2021-07-19 Thread Yugo NAGATA
On Tue, 13 Jul 2021 11:59:49 +0900 Yugo NAGATA wrote: > Hello, > > During reading the documentation of libpq [1] , I found the following > description: > > In the nonblocking state, calls to PQsendQuery, PQputline, PQputnbytes, > PQputCopyData, and PQendcopy will not block but instead return

Re: Addition of authenticated ID to pg_stat_activity

2021-07-19 Thread Aleksander Alekseev
Hi Michael, > Just to make the discussion move on, attached is an updated version > doing that. The code seems OK, but I have mixed feelings about the way that the VIEW currently works. Here is what I get when a single user is connected via a UNIX socket: 43204 (master) =# select * from

Re: speed up verifying UTF-8

2021-07-19 Thread Vladimir Sitnikov
Thank you, It looks like it is important to have shrx for x86 which appears only when -march=x86-64-v3 is used (see https://github.com/golang/go/issues/47120#issuecomment-877629712 ). Just in case: I know x86 wound not use fallback implementation, however, the sole purpose of shift-based DFA is

Re: row filtering for logical replication

2021-07-19 Thread Tomas Vondra
On 7/19/21 1:30 PM, Amit Kapila wrote: > On Wed, Jul 14, 2021 at 8:03 PM Tomas Vondra > wrote: >> >> On 7/14/21 2:50 PM, Amit Kapila wrote: >>> On Wed, Jul 14, 2021 at 3:58 PM Tomas Vondra >>> >>> I think apart from the above, it might be good if we can find what >>> some other databases does in

Re: row filtering for logical replication

2021-07-19 Thread Tomas Vondra
On 7/19/21 1:00 PM, Dilip Kumar wrote: > On Mon, Jul 19, 2021 at 3:12 PM Amit Kapila wrote: >> a. Just log it and move to the next row >> b. send to stats collector some info about this which can be displayed >> in a view and then move ahead >> c. just skip it like any other row that doesn't

Re: Re[3]: On login trigger: take three

2021-07-19 Thread Greg Nancarrow
On Mon, Jul 19, 2021 at 8:18 PM Ibrar Ahmed wrote: > > The patch does not apply, and rebase is required. > Attached a rebased patch (minor updates to the test code). Regards, Greg Nancarrow Fujitsu Australia v17-0001-on_client_connect_event_trigger.patch Description: Binary data

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-19 Thread James Coleman
On Sat, Jul 17, 2021 at 4:36 AM David Rowley wrote: > > On Sat, 17 Jul 2021 at 01:14, James Coleman wrote: > > The only remaining question I have is whether or not costing needs to > > change, given the very significant speedup for datum sort. > > I'm looking at cost_tuplesort and the only

Re: refactoring basebackup.c

2021-07-19 Thread tushar
On 7/16/21 12:43 PM, Dilip Kumar wrote: I think the problem is that bbsink_gzip_end_archive() is not forwarding the end request to the next bbsink. The attached patch so fix it. Thanks Dilip. Reported issue seems to be fixed now with your patch [edb@centos7tushar bin]$ ./pg_basebackup

Re: O_DIRECT on macOS

2021-07-19 Thread John Naylor
On Mon, Jul 19, 2021 at 12:55 AM Thomas Munro wrote: > > On Mon, Jul 19, 2021 at 4:42 PM Tom Lane wrote: > > prairiedog thinks that Assert is too optimistic about whether all > > those flags exist. > > Fixed. > > (Huh, I received no -committers email for 2dbe8905.) It didn't show up in the

Re: .ready and .done files considered harmful

2021-07-19 Thread Dipesh Pandit
Hi, > I agree, I missed this part. The .history file should be given higher preference. > I will take care of it in the next patch. Archiver does not have access to shared memory and the current timeline ID is not available at archiver. In order to keep track of timeline switch we have to push a

RE: Skipping logical replication transactions on subscriber side

2021-07-19 Thread houzj.f...@fujitsu.com
On July 19, 2021 2:40 PM Masahiko Sawada wrote: > I've attached the updated version patch that incorporated all comments > I got so far except for the clearing error details part I mentioned > above. After getting a consensus on those parts, I'll incorporate the > idea into the patches. Hi

Re: 2021-07 CF now in progress

2021-07-19 Thread Ibrar Ahmed
On Mon, Jul 12, 2021 at 4:59 PM Ibrar Ahmed wrote: > > Hackers, > > The Commitfest 2021-07 is now in progress. It is one of the biggest one. > Total number of patches of this commitfest is 342. > > Needs review: 204. > Waiting on Author: 40. > Ready for Committer: 18. > Committed: 57. > Moved to

Re: refactoring basebackup.c

2021-07-19 Thread Dilip Kumar
On Fri, Jul 16, 2021 at 12:43 PM Dilip Kumar wrote: > > On Mon, Jul 12, 2021 at 5:51 PM tushar wrote: > > > > On 7/8/21 9:26 PM, Robert Haas wrote: > > > Here at last is a new version. > > Please refer this scenario ,where backup target using > > --server-compression is closing the server > >

Re: row filtering for logical replication

2021-07-19 Thread Amit Kapila
On Wed, Jul 14, 2021 at 8:03 PM Tomas Vondra wrote: > > On 7/14/21 2:50 PM, Amit Kapila wrote: > > On Wed, Jul 14, 2021 at 3:58 PM Tomas Vondra > > > > I think apart from the above, it might be good if we can find what > > some other databases does in this regard? > > > > Yeah, that might tell us

Re: badly calculated width of emoji in psql

2021-07-19 Thread Laurenz Albe
On Mon, 2021-07-19 at 16:46 +0900, Michael Paquier wrote: > > In your opinion, would the current one-line patch proposal make things > > strictly better than they are today, or would it have mixed results? > > I'm wondering how to help this patch move forward for the current > > commitfest, or if

Re: refactoring basebackup.c

2021-07-19 Thread tushar
On 7/8/21 9:26 PM, Robert Haas wrote: Here at last is a new version. if i try to perform pg_basebackup using "-t server " option against localhost V/S remote machine , i can see difference in backup size. data directory whose size is [edb@centos7tushar bin]$ du -sch data/ 578M    data/ 578M  

Re: row filtering for logical replication

2021-07-19 Thread Dilip Kumar
On Mon, Jul 19, 2021 at 3:12 PM Amit Kapila wrote: > a. Just log it and move to the next row > b. send to stats collector some info about this which can be displayed > in a view and then move ahead > c. just skip it like any other row that doesn't match the filter clause. > > I am not sure if

Re: Improve join selectivity estimation using extended statistics

2021-07-19 Thread Ibrar Ahmed
On Mon, Mar 15, 2021 at 8:42 PM Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > > > On 11.03.2021 03:47, Tomas Vondra wrote: > > Hi Konstantin, > > > > Thanks for working on this! Using extended statistics to improve join > > cardinality estimates was definitely on my radar, and this

Re: shared-memory based stats collector

2021-07-19 Thread Ibrar Ahmed
On Wed, Apr 7, 2021 at 8:05 AM Kyotaro Horiguchi wrote: > At Tue, 6 Apr 2021 09:32:16 -0700, Andres Freund > wrote in > > Hi, > > > > On 2021-04-05 02:29:14 -0700, Andres Freund wrote: > .. > > I'm inclined to push patches > > [PATCH v60 05/17] pgstat: split bgwriter and checkpointer messages.

Re: Column Filtering in Logical Replication

2021-07-19 Thread Ibrar Ahmed
On Tue, Jul 13, 2021 at 7:44 PM Rahila Syed wrote: > > Hi Tomas, > > Thank you for your comments. > > >> >> > >> > Currently, this capability is not included in the patch. If the table on >> > the subscriber >> > server has lesser attributes than that on the publisher server, it >> > throws an

Re: Re[3]: On login trigger: take three

2021-07-19 Thread Ibrar Ahmed
On Wed, Jul 7, 2021 at 5:55 AM Greg Nancarrow wrote: > On Sun, Jul 4, 2021 at 1:21 PM vignesh C wrote: > > > > CFBot shows the following failure: > > # poll_query_until timed out executing this query: > > # SELECT '0/3046250' <= replay_lsn AND state = 'streaming' FROM > >

Re: Minimal logical decoding on standbys

2021-07-19 Thread Ibrar Ahmed
On Fri, Jul 16, 2021 at 1:07 PM Drouvot, Bertrand wrote: > Hi Andres, > > On 6/22/21 12:38 PM, Drouvot, Bertrand wrote: > > Hi Andres, > > > > On 6/14/21 7:41 AM, Drouvot, Bertrand wrote: > >> Hi Andres, > >> > >> On 4/8/21 5:47 AM, Andres Freund wrote: > >>> Hi, > >>> > >>> On 2021-04-07

Re: badly calculated width of emoji in psql

2021-07-19 Thread Pavel Stehule
po 19. 7. 2021 v 9:46 odesílatel Michael Paquier napsal: > On Wed, Jul 07, 2021 at 06:03:34PM +, Jacob Champion wrote: > > I would guess that's the key issue here. If we choose a particular > > width for emoji characters, is there anything keeping a terminal's font > > from doing something

Re: corruption of WAL page header is never reported

2021-07-19 Thread Yugo NAGATA
On Mon, 19 Jul 2021 06:32:28 -0300 Ranier Vilela wrote: > Em seg., 19 de jul. de 2021 às 06:15, Yugo NAGATA > escreveu: > > > On Mon, 19 Jul 2021 17:47:07 +0900 (JST) > > Kyotaro Horiguchi wrote: > > > > > At Mon, 19 Jul 2021 16:00:39 +0900, Yugo NAGATA > > wrote in > > > > Your patch

postgresql.conf.sample missing units

2021-07-19 Thread Pavel Luzanov
Hello, I found that the start section of the postgresql.conf file is missing a description of two units: bytes (appeared in version 11) and microseconds (in version 12). The attached patch makes these changes to the postgresql.conf.sample file. -- Pavel Luzanov Postgres Professional:

Re: row filtering for logical replication

2021-07-19 Thread Amit Kapila
On Sat, Jul 17, 2021 at 3:05 AM Alvaro Herrera wrote: > > On 2021-Jul-16, Greg Nancarrow wrote: > > > On Fri, Jul 16, 2021 at 3:50 PM Amit Kapila wrote: > > > > > > I am not so sure about different filters for old and new rows but it > > > makes sense to by default apply the filter to both old

Re: Replace remaining castNode(…, lfirst(…)) and friends calls with l*_node()

2021-07-19 Thread Dagfinn Ilmari Mannsåker
Peter Eisentraut writes: > On 15.07.21 13:12, Dagfinn Ilmari Mannsåker wrote: >> ilm...@ilmari.org (Dagfinn Ilmari Mannsåker) writes: >> >>> Hi hackers, >>> >>> We've had node-casting versions of the list extraction macros since >>> 2017, but several cases of the written-out version have been

Re: corruption of WAL page header is never reported

2021-07-19 Thread Ranier Vilela
Em seg., 19 de jul. de 2021 às 06:15, Yugo NAGATA escreveu: > On Mon, 19 Jul 2021 17:47:07 +0900 (JST) > Kyotaro Horiguchi wrote: > > > At Mon, 19 Jul 2021 16:00:39 +0900, Yugo NAGATA > wrote in > > > Your patch doesn't fix the issue that the error message is never > reported in > > > standby

Re: corruption of WAL page header is never reported

2021-07-19 Thread Yugo NAGATA
On Mon, 19 Jul 2021 17:47:07 +0900 (JST) Kyotaro Horiguchi wrote: > At Mon, 19 Jul 2021 16:00:39 +0900, Yugo NAGATA wrote > in > > Your patch doesn't fix the issue that the error message is never reported in > > standby mode. When a WAL page header is broken, the standby would silently > >

Re: Added schema level support for publication.

2021-07-19 Thread Greg Nancarrow
On Fri, Jul 16, 2021 at 8:13 PM vignesh C wrote: > > Modified. > > Thanks for the comments, these issues are fixed as part of the v12 patch > posted at [1]. > [1] - > https://www.postgresql.org/message-id/CALDaNm3V9ny5dJM8nofLGJ3zDuDG0gS2dX%2BAhDph--U5y%2B4VbQ%40mail.gmail.com > There seems

Re: Why ALTER SUBSCRIPTION ... SET (slot_name='none') requires subscription disabled?

2021-07-19 Thread Amit Kapila
On Fri, Jul 16, 2021 at 2:12 PM Japin Li wrote: > > > On Fri, 16 Jul 2021 at 14:06, Amit Kapila wrote: > > On Fri, Jul 9, 2021 at 8:20 AM Japin Li wrote: > >> > >> On Thu, 08 Jul 2021 at 18:17, Amit Kapila wrote: > >> > On Thu, Jul 8, 2021 at 3:43 PM Japin Li wrote: > >> > >> Please consider

Re: [HACKERS] logical decoding of two-phase transactions

2021-07-19 Thread Amit Kapila
On Mon, Jul 19, 2021 at 1:00 PM Peter Smith wrote: > > On Mon, Jul 19, 2021 at 4:41 PM Amit Kapila wrote: > > > > > > > > OK. I have implemented this reported [1] potential buffer overrun > > > using the constraining strlcpy, because the GID limitation of 200 > > > bytes is already mentioned in

Re: corruption of WAL page header is never reported

2021-07-19 Thread Kyotaro Horiguchi
me> Howeer, I'm still on the opinion that we don't need to check that me> while in standby mode. Of course it's typo of "while not in standby mode". sorry.. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Skipping logical replication transactions on subscriber side

2021-07-19 Thread Amit Kapila
On Mon, Jul 19, 2021 at 12:10 PM Masahiko Sawada wrote: > > On Sat, Jul 17, 2021 at 12:02 AM Masahiko Sawada > wrote: > > > > 1. How to clear apply worker errors. IIUC we've discussed that once > > the apply worker skipped the transaction we leave the error entry > > itself but clear its fields

Re: corruption of WAL page header is never reported

2021-07-19 Thread Kyotaro Horiguchi
At Mon, 19 Jul 2021 16:00:39 +0900, Yugo NAGATA wrote in > Your patch doesn't fix the issue that the error message is never reported in > standby mode. When a WAL page header is broken, the standby would silently > repeat > retrying forever. Ok, I see your point and agree to that. > I think

Re: Update maintenance_work_mem/autovacuum_work_mem to reflect the 1GB limitation with VACUUM

2021-07-19 Thread Masahiko Sawada
On Wed, Jul 7, 2021 at 8:44 PM David Rowley wrote: > > On Sun, 4 Jul 2021 at 22:38, David Rowley wrote: > > I could do with a 2nd opinion about if we should just adjust the > > maximum value for the autovacuum_work_mem GUC to 1GB in master. > > > > I'm also not sure if since we'd not backpatch

Re: Toast compression method options

2021-07-19 Thread Dilip Kumar
On Sun, Jul 18, 2021 at 9:15 PM Dilip Kumar wrote: > > On Wed, Jul 14, 2021 at 5:35 PM vignesh C wrote: > > > > On Thu, May 6, 2021 at 7:24 PM Dilip Kumar wrote > > > > The patch does not apply on Head anymore, could you rebase and post a > > patch. I'm changing the status to "Waiting for

Re: Failure with 004_logrotate in prairiedog

2021-07-19 Thread Michael Paquier
On Mon, Jul 19, 2021 at 04:15:36PM +0900, Kyotaro Horiguchi wrote: > When rotation happens, the metainfo file is once removed then > created. If slurp_file in the metafile-checking loop hits the gap, the > slurp_file fails with ENOENT. I can read the following code, as of

Re: badly calculated width of emoji in psql

2021-07-19 Thread Michael Paquier
On Wed, Jul 07, 2021 at 06:03:34PM +, Jacob Champion wrote: > I would guess that's the key issue here. If we choose a particular > width for emoji characters, is there anything keeping a terminal's font > from doing something different anyway? I'd say that we are doing our best in guessing

Re: [HACKERS] logical decoding of two-phase transactions

2021-07-19 Thread Peter Smith
On Mon, Jul 19, 2021 at 4:41 PM Amit Kapila wrote: > > On Mon, Jul 19, 2021 at 9:19 AM Peter Smith wrote: > > > > On Mon, Jul 19, 2021 at 12:43 PM Amit Kapila > > wrote: > > > > > > On Mon, Jul 19, 2021 at 1:55 AM Tom Lane wrote: > > > > > > > > Amit Kapila writes: > > > > > Pushed. > > > >

Re: alter table set TABLE ACCESS METHOD

2021-07-19 Thread Michael Paquier
On Thu, Jul 15, 2021 at 10:49:23PM -0500, Justin Pryzby wrote: > Also, there were two redundant checks for multiple SET ACCESS METHOD commands. > But one of them wasn't hit if the ALTER was setting the current AM due to the > no-op test. Yep. > I think it's better to fail in every case, and not

Re: Failure with 004_logrotate in prairiedog

2021-07-19 Thread Kyotaro Horiguchi
At Sun, 18 Jul 2021 12:32:20 +0900, Michael Paquier wrote in > Hi all, > > prairiedog has failed in a way that seems a bit obscure to me: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prairiedog=2021-07-18%2000%3A23%3A29 > > Here are the details of the failure: > server signaled

Re: Introduce pg_receivewal gzip compression tests

2021-07-19 Thread Michael Paquier
On Fri, Jul 16, 2021 at 02:08:57PM +0900, Michael Paquier wrote: > This behavior is rather debatable, and it would be more instinctive to > me to just skip any business related to the pre-padding if compression > is enabled, at the cost of one extra callback in WalWriteMethod to > grab the

Re: corruption of WAL page header is never reported

2021-07-19 Thread Yugo NAGATA
On Mon, 19 Jul 2021 15:14:41 +0900 (JST) Kyotaro Horiguchi wrote: > Hello. > > At Sun, 18 Jul 2021 04:55:05 +0900, Yugo NAGATA wrote > in > > Hello, > > > > I found that any corruption of WAL page header found during recovery is > > never > > reported in log messages. If wal page header is

Re: automatically generating node support functions

2021-07-19 Thread Peter Eisentraut
On 07.06.21 22:27, Peter Eisentraut wrote: I wrote a script to automatically generate the node support functions (copy, equal, out, and read, as well as the node tags enum) from the struct definitions. The first eight patches are to clean up various inconsistencies to make parsing or

Re: [HACKERS] logical decoding of two-phase transactions

2021-07-19 Thread Amit Kapila
On Mon, Jul 19, 2021 at 9:19 AM Peter Smith wrote: > > On Mon, Jul 19, 2021 at 12:43 PM Amit Kapila wrote: > > > > On Mon, Jul 19, 2021 at 1:55 AM Tom Lane wrote: > > > > > > Amit Kapila writes: > > > > Pushed. > > > > > > I think you'd be way better off making the gid fields be "char *" > > >

Re: Skipping logical replication transactions on subscriber side

2021-07-19 Thread Masahiko Sawada
On Sat, Jul 17, 2021 at 12:02 AM Masahiko Sawada wrote: > > On Wed, Jul 14, 2021 at 5:14 PM Masahiko Sawada wrote: > > > > On Mon, Jul 12, 2021 at 8:52 PM Amit Kapila wrote: > > > > > > On Mon, Jul 12, 2021 at 11:13 AM Masahiko Sawada > > > wrote: > > > > > > > > On Mon, Jul 12, 2021 at 1:15

Re: Skipping logical replication transactions on subscriber side

2021-07-19 Thread Masahiko Sawada
On Mon, Jul 19, 2021 at 2:22 PM Amit Kapila wrote: > > On Fri, Jul 16, 2021 at 8:33 PM Masahiko Sawada wrote: > > > > On Wed, Jul 14, 2021 at 5:14 PM Masahiko Sawada > > wrote: > > > > > > Sounds good. I'll incorporate this in the next version patch that I'm > > > planning to submit this week.

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2021-07-19 Thread Ronan Dunklau
> Looks like I did a sloppy job of that. I messed up the condition in > standard_qp_callback() that sets the ORDER BY aggregate pathkeys so > that it accidentally set them when there was an unsortable GROUP BY > clause, as highlighted by the postgres_fdw tests failing. I've now > added a comment

Re: Replace remaining castNode(…, lfirst(…)) and friends calls with l*_node()

2021-07-19 Thread Peter Eisentraut
On 15.07.21 13:12, Dagfinn Ilmari Mannsåker wrote: ilm...@ilmari.org (Dagfinn Ilmari Mannsåker) writes: Hi hackers, We've had node-casting versions of the list extraction macros since 2017, but several cases of the written-out version have been added since then (even by Tom, who added the

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-19 Thread Masahiko Sawada
Sorry for the late reply. On Sat, Jul 10, 2021 at 11:55 AM Andres Freund wrote: > > Hi, > > On 2021-07-09 10:17:49 -0700, Andres Freund wrote: > > On 2021-07-07 20:46:38 +0900, Masahiko Sawada wrote: > > > Currently, the TIDs of dead tuples are stored in an array that is > > > collectively

Re: corruption of WAL page header is never reported

2021-07-19 Thread Kyotaro Horiguchi
Hello. At Sun, 18 Jul 2021 04:55:05 +0900, Yugo NAGATA wrote in > Hello, > > I found that any corruption of WAL page header found during recovery is never > reported in log messages. If wal page header is broken, it is detected in > XLogReaderValidatePageHeader called from XLogPageRead, but

Re: RFC: Logging plan of the running query

2021-07-19 Thread Fujii Masao
On 2021/07/19 11:28, torikoshia wrote: Agreed. Updated the patch. Thanks for updating the patch! +bool +SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason) I don't think that procsignal.c is proper place to check the permission and check whether the specified PID indicates a

  1   2   >