Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-20 Thread Andres Freund
Hi, On 2022-02-21 12:39:31 +0530, Amit Kapila wrote: > Fair enough. Then, how about the following keeping the following information: Mostly sounds good. > * subid (subscription id) > * subname (subscription name) Coming from catalog, via join, I assume? > * sync_error_count/sync_failure_coun

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-02-20 Thread Andy Fan
> > > >> It actually deals with a more general form of this case, because the > >> clauses don't need to reference the same attribute - so for example this > >> would work too, assuming there is extended stats object on the columns > >> on each side: > >> > >> P(A.c = B.d | (A.e < 42) & (B.f < 42

RE: Synchronizing slots from primary to standby

2022-02-20 Thread kato-...@fujitsu.com
Hello, This patch status is already returned with feedback. However, I've applied this patch on 27b77ecf9f and tested so report it. make installcheck-world is passed. However, when I promote the standby server and update on the new primary server, apply worker could not start logical replication

Re: make tuplestore helper function

2022-02-20 Thread Michael Paquier
On Thu, Feb 17, 2022 at 04:10:01PM +0900, Michael Paquier wrote: > Asserting that we are in the correct memory context in when calling > MakeFuncResultTuplestore() sounds rather sensible from here as per the > magics done in the various json functions. Still, it really feels > like we could do a m

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-02-20 Thread Andy Fan
Thanks for the detailed explanation. On Sat, Feb 19, 2022 at 2:27 AM Robert Haas wrote: > On Fri, Feb 18, 2022 at 12:56 AM Andy Fan > wrote: > > What do you think about moving on this feature? The items known by me > > are: 1). Make sure the estimation error can be fixed or discuss if my > cu

Re: Slow standby snapshot

2022-02-20 Thread Michail Nikolaev
Hello, Andrey. Thanks for your efforts. > Patch on barrier seems too complicated to me right now. I’d propose to focus > on KnowAssignedXidsNext patch: it’s clean, simple and effective. I'll extract it to the separated patch later. > I’ve rebased the patch so that it does not depend on previous

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-20 Thread Amit Kapila
On Mon, Feb 21, 2022 at 11:04 AM Andres Freund wrote: > > On 2022-02-21 12:56:46 +0900, Masahiko Sawada wrote: > > > To take a precedent, we used to store accumulative statistics such as > > spill_txns to pg_stat_replication, but then for the same reason we moved > > those statistics to the new st

Re: Assert in pageinspect with NULL pages

2022-02-20 Thread Alexander Lakhin
Hello Michael, 18.02.2022 06:07, Michael Paquier wrpte: > On Thu, Feb 17, 2022 at 09:00:20PM -0600, Justin Pryzby wrote: >> BRIN can also crash if passed a non-brin index. >> >> I've been sitting on this one for awhile. Feel free to include it in your >> patchset. > Ugh. Thanks! I am keeping a

Re: Optionally automatically disable logical replication subscriptions on error

2022-02-20 Thread Peter Smith
Thanks for addressing my previous comments. Now I have looked at v19. On Mon, Feb 21, 2022 at 11:25 AM osumi.takami...@fujitsu.com wrote: > > On Friday, February 18, 2022 3:27 PM Peter Smith > wrote: > > Hi. Below are my code review comments for v18. > Thank you for your review ! ... > > 5. src

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-02-20 Thread Etsuro Fujita
On Sat, Feb 19, 2022 at 6:55 AM David Zhang wrote: > Tried to apply the patches to master branch, no warning found and > regression test passed. Thanks for testing! > Now, we have many places (5) calling the same function with a constant > number 3. Is this a good time to consider redefine t

Re: Separate the result of \watch for each query execution (psql)

2022-02-20 Thread Pavel Stehule
Hi po 21. 2. 2022 v 6:19 odesílatel Noboru Saito napsal: > I need a way to separate the results of \watch for each query execution. > > There is only a blank line between the results of \watch. > However, there is also a blank line after the title, which complicates > the rules. > > My suggestio

Re: pg_stat_get_replication_slot and pg_stat_get_subscription_worker incorrectly marked as proretset

2022-02-20 Thread Masahiko Sawada
Hi, On Mon, Feb 21, 2022 at 2:36 PM Michael Paquier wrote: > > Hi all, > (Author and committer added in CC.) > > While reviewing the code of a bunch of SRF functions in the core code, > I have noticed that the two functions mentioned in $subject are marked > as proretset but both functions don't

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-02-20 Thread Etsuro Fujita
On Fri, Feb 18, 2022 at 1:46 AM Fujii Masao wrote: > I reviewed 0001 patch. It looks good to me except the following minor things. > If these are addressed, I think that the 001 patch can be marked as ready for > committer. OK > +* Also determine to commit (sub)transactions opened on t

pg_stat_get_replication_slot and pg_stat_get_subscription_worker incorrectly marked as proretset

2022-02-20 Thread Michael Paquier
Hi all, (Author and committer added in CC.) While reviewing the code of a bunch of SRF functions in the core code, I have noticed that the two functions mentioned in $subject are marked as proretset but both functions don't return a set of tuples, just one record for the object given in input. It

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-20 Thread Andres Freund
Hi, On 2022-02-21 12:56:46 +0900, Masahiko Sawada wrote: > > The patch you referenced [1] should just store the stats in the > > pg_stat_subscription view, not pg_stat_subscription_workers. > > > > It *does* make sense to keep stats about the number of table syncs that > > failed > > etc. But tha

Separate the result of \watch for each query execution (psql)

2022-02-20 Thread Noboru Saito
I need a way to separate the results of \watch for each query execution. There is only a blank line between the results of \watch. However, there is also a blank line after the title, which complicates the rules. My suggestion is to insert a "form feed(\f)" (preferably a newline) before the resul

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-20 Thread Amit Kapila
On Sat, Feb 19, 2022 at 10:35 PM David G. Johnston wrote: > > On Sat, Feb 19, 2022 at 9:37 AM Andres Freund wrote: >> >> IMO the type of information you'd want for apply failures is substantially >> >> different enough from worker failures that I don't really see the temptation >> to put them in

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-20 Thread Amit Kapila
On Sat, Feb 19, 2022 at 10:07 PM Andres Freund wrote: > > > I also still think that _worker shouldn't be part of any of the naming > here. > Okay, the other options that comes to mind for this are: pg_subscription_replication_error, or pg_subscription_lreplication_error, or pg_subscription_lrep_e

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-20 Thread Masahiko Sawada
On Sun, Feb 20, 2022 at 1:02 AM Andres Freund wrote: > > Hi, > > On 2022-02-19 22:38:19 +0900, Masahiko Sawada wrote: > > On Sat, Feb 19, 2022 at 5:32 AM Andres Freund wrote: > > > On 2022-02-18 17:26:04 +0900, Masahiko Sawada wrote: > > > > With this change, pg_stat_subscription_workers will be

RE: Failed transaction statistics to measure the logical replication progress

2022-02-20 Thread osumi.takami...@fujitsu.com
On Saturday, February 19, 2022 12:00 AM osumi.takami...@fujitsu.com wrote: > On Friday, February 18, 2022 3:34 PM Tang, Haiying/唐 海英 > wrote: > > On Wed, Jan 12, 2022 8:35 PM osumi.takami...@fujitsu.com > > wrote: > > 4) I noticed that the abort_count doesn't include aborted streaming > > trans

Re: Fix overflow in DecodeInterval

2022-02-20 Thread Joseph Koshakow
On Sun, Feb 20, 2022 at 6:37 PM Tom Lane wrote: > Couple of quick comments: Thanks for the comments Tom, I'll work on fixing these and submit a new patch. > * The uses of tm2itm make me a bit itchy. Is that sweeping > upstream-of-there overflow problems under the rug? I agree, I'm not super ha

Re: Trap errors from streaming child in pg_basebackup to exit early

2022-02-20 Thread Michael Paquier
On Fri, Feb 18, 2022 at 10:00:43PM +0100, Daniel Gustafsson wrote: > This is good idea, I was going in a different direction earlier with a test > but > this is cleaner. The attached 0001 refactors pump_until; 0002 fixes a trivial > spelling error found while hacking; and 0003 is the previous pat

Re: Uniforms the errors msgs at tuplestore paths

2022-02-20 Thread Ranier Vilela
Em dom., 20 de fev. de 2022 às 22:08, Michael Paquier escreveu: > On Sun, Feb 20, 2022 at 11:37:33AM -0300, Ranier Vilela wrote: > > I can't see: > > plperl.c > > pl_exec.c > > pttcl.c > > > > Only jsonfuncs.c, but the error about "materialized mode" is not > reported. > > Melanie has done a nice

Re: Uniforms the errors msgs at tuplestore paths

2022-02-20 Thread Michael Paquier
On Sun, Feb 20, 2022 at 11:37:33AM -0300, Ranier Vilela wrote: > I can't see: > plperl.c > pl_exec.c > pttcl.c > > Only jsonfuncs.c, but the error about "materialized mode" is not reported. Melanie has done a nice analysis of all the code paths doing materialization checks for her patch with SRF

Re: pg_upgrade verbosity when redirecting output to log file

2022-02-20 Thread Andres Freund
Hi, On 2022-02-18 19:46:26 -0600, Justin Pryzby wrote: > +* If outputting to a tty / or , append newline. pg_log_v() will put > the > +* individual progress items onto the next line. > +*/ > + if (log_opts.isatty || log_opts.verbose) > > I guess the comment should sa

Re: pgsql: Add support for building with ZSTD.

2022-02-20 Thread Michael Paquier
On Sun, Feb 20, 2022 at 09:45:05AM -0500, Robert Haas wrote: > No issues at all with you adjusting this, but I think that sentence > reads a little awkwardly. Thanks. > Perhaps instead of "The default is > zstd, that would be the command found in > PATH." you could write something like "The defau

Re: WIP: System Versioned Temporal Table

2022-02-20 Thread Corey Huinker
> > > The spec does not allow schema changes at all on a a system versioned > table, except to change the system versioning itself. > > That would greatly simplify things!

Re: set TESTDIR from perl rather than Makefile

2022-02-20 Thread Andres Freund
Hi, On 2022-02-19 17:53:09 -0600, Justin Pryzby wrote: > I also meant to also attach it. Is the patch actually independent of the other patches in your stack? I like this concept a lot: - I've had to use a wrapper around individual tap tests for meson, just to set the CWD etc. - Being able t

RE: Optionally automatically disable logical replication subscriptions on error

2022-02-20 Thread osumi.takami...@fujitsu.com
On Friday, February 18, 2022 3:27 PM Peter Smith wrote: > Hi. Below are my code review comments for v18. Thank you for your review ! > == > > 1. Commit Message - wording > > BEFORE > To partially remedy the situation, adding a new subscription_parameter named > 'disable_on_error'. > >

Re: Fix overflow in DecodeInterval

2022-02-20 Thread Tom Lane
Joseph Koshakow writes: > Attached is a patch of my first pass. The to_char method isn't finished > and I need to add a bunch of tests, but everything else is in place. It > ended up being a fairly large change in case anyone wants to take a look > the changes so far. Couple of quick comments: *

Re: do only critical work during single-user vacuum?

2022-02-20 Thread Peter Geoghegan
On Sun, Feb 20, 2022 at 2:15 PM Andres Freund wrote: > We could e.g. add an error if FreezeMultiXactId() needs to create a new > multixact for a far-in-the-past xid. That's not great, of course, but if we > include the precise cause (pid of backend / prepared xact name / slot name / > ...) necessi

Re: do only critical work during single-user vacuum?

2022-02-20 Thread Noah Misch
On Sun, Feb 20, 2022 at 02:15:37PM -0800, Andres Freund wrote: > On 2022-02-19 20:57:57 -0800, Noah Misch wrote: > > On Wed, Feb 16, 2022 at 03:43:12PM +0700, John Naylor wrote: > > > On Wed, Feb 16, 2022 at 6:17 AM Peter Geoghegan wrote: > > > > On Tue, Feb 15, 2022 at 9:28 AM Peter Geoghegan wr

Re: do only critical work during single-user vacuum?

2022-02-20 Thread Andres Freund
Hi, On 2022-02-19 20:57:57 -0800, Noah Misch wrote: > On Wed, Feb 16, 2022 at 03:43:12PM +0700, John Naylor wrote: > > On Wed, Feb 16, 2022 at 6:17 AM Peter Geoghegan wrote: > > > On Tue, Feb 15, 2022 at 9:28 AM Peter Geoghegan wrote: > > > > > > I did notice from my own testing of the failsafe

Re: initdb / bootstrap design

2022-02-20 Thread Andres Freund
Hi, On 2022-02-19 20:46:26 -0500, Tom Lane wrote: > I tried it like that (full patch attached) and the results are intensely > disappointing. On my Mac laptop, the time needed for 50 iterations of > initdb drops from 16.8 sec to 16.75 sec. Hm. I'd hoped for at least a little bit bigger win. But

Re: Adding CI to our tree (ccache)

2022-02-20 Thread Justin Pryzby
On Sun, Feb 20, 2022 at 12:47:31PM -0800, Andres Freund wrote: > > Did you ever try to use clcache (or others) ? > > > > When I tried, it refused to cache because of our debug settings > > (DebugInformationFormat) - which seem to be enabled even in release mode. > > > I wonder if that'll be an is

Re: killing perl2host

2022-02-20 Thread Thomas Munro
On Sun, Feb 20, 2022 at 10:51 AM Andrew Dunstan wrote: > On 2/19/22 16:34, Andres Freund wrote: > >> The first > >> removes perl2host completely and adjusts all the places that use it. > >> The second removes almost all the remaining msys special processing in > >> TAP tests. Very nice improveme

Re: Adding CI to our tree (ccache)

2022-02-20 Thread Andres Freund
Hi, On 2022-02-20 13:36:55 -0600, Justin Pryzby wrote: > Have you tried to use the yet-to-be-released ccache with MSVC ? Yes, it doesn't work, because it requires cl.exe to be used in a specific way (only a single input file, specific output file naming). Which would require a decent amount of ch

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-02-20 Thread Peter Geoghegan
, On Sun, Feb 20, 2022 at 7:30 AM Robert Haas wrote: > Right, so we at least need to add a similar comment to what I proposed > for MXIDs, and maybe other changes are needed, too. Agreed. > > Maybe the way that FreezeLimit/cutoff_xid is overloaded can be fixed > > here, to make all of this less

Re: libpq async duplicate error results

2022-02-20 Thread Tom Lane
Alvaro Herrera writes: > On 2022-Feb-07, Tom Lane wrote: >> I see that PQgetResult does a hacky "resetPQExpBuffer(&conn->errorMessage)" >> in between pipelined queries. It seems like if there are real usability >> issues in this area, they probably come from that not being well placed. >> In part

Re: Adding CI to our tree (ccache)

2022-02-20 Thread Justin Pryzby
Have you tried to use the yet-to-be-released ccache with MSVC ? Also, do you know about msbuild /outputResultsCache ? When I tried that, it gave a bunch of error. https://cirrus-ci.com/task/5697497241747456 |[16:35:13.605] 1>c:\cirrus\pgsql.sln.metaproj : error : MSB4252: Project "c:\cirru

Re: Slow standby snapshot

2022-02-20 Thread Andrey Borodin
> On 22 Nov 2021, at 14:05, Michail Nikolaev wrote: > >> Write barrier must be issued after write, not before. >> Don't we need to issue read barrier too? > > The write barrier is issued after the changes to KnownAssignedXidsNext > and KnownAssignedXidsValid arrays and before the update of > h

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-02-20 Thread Robert Haas
On Fri, Feb 18, 2022 at 7:12 PM Peter Geoghegan wrote: > We have to worry about XIDs from MultiXacts (and xmax values more > generally). And we have to worry about the case where we start out > with only xmin frozen (by an earlier VACUUM), and then have to freeze > xmax too. I believe that we have

Re: Print warning when I execute my own extension function

2022-02-20 Thread Dong Wook Lee
now I found a bug in this code. I found the code of this part below is wrong ``` char *buffer = palloc(7 * sizeof(char)); >> unsigned int offset = sizeof(buffer); // wrong buffer[--offset] = '\0'; /* here */ ``` Thank you for replying 2022년 2월 20일 (일) 오후 11:51, Dong Wook Lee 님이 작성: > > I found a

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-02-20 Thread Robert Haas
On Sat, Feb 19, 2022 at 8:54 PM Andres Freund wrote: > > Leaving behind disconnected/orphaned heap-only tuples is pretty much > > pointless anyway, since they'll never be accessible by index scans. > > Even after a REINDEX, since there is no root item from the heap page > > to go in the index. (A

Re: Print warning when I execute my own extension function

2022-02-20 Thread Dong Wook Lee
I found a source code line that prints warning in the source code (src/backend/utils/mmgr/aset.c, line 1496) /* │ 1492 * Check for overwrite of padding space in an allocated chunk. │ 1493

Re: pgsql: Add support for building with ZSTD.

2022-02-20 Thread Robert Haas
On Sat, Feb 19, 2022 at 1:19 AM Michael Paquier wrote: > On Fri, Feb 18, 2022 at 06:53:10PM +, Robert Haas wrote: > > Add support for building with ZSTD. > > > > This commit doesn't actually add anything that uses ZSTD; that will be > > done separately. It just puts the basic infrastructure in

Re: Uniforms the errors msgs at tuplestore paths

2022-02-20 Thread Ranier Vilela
Em dom., 20 de fev. de 2022 às 11:30, Justin Pryzby escreveu: > On Sun, Feb 20, 2022 at 11:12:42AM -0300, Ranier Vilela wrote: > > Like how the commit > > > https://github.com/postgres/postgres/commit/07daca53bfcad59618a9c6fad304e380cc9d2bc1 > > The are some paths that were missed: > > I think th

Re: Uniforms the errors msgs at tuplestore paths

2022-02-20 Thread Justin Pryzby
On Sun, Feb 20, 2022 at 11:12:42AM -0300, Ranier Vilela wrote: > Like how the commit > https://github.com/postgres/postgres/commit/07daca53bfcad59618a9c6fad304e380cc9d2bc1 > The are some paths that were missed: I think these are all unified by the existing tuplestore patch. https://commitfest.post

Uniforms the errors msgs at tuplestore paths

2022-02-20 Thread Ranier Vilela
Hi, Like how the commit https://github.com/postgres/postgres/commit/07daca53bfcad59618a9c6fad304e380cc9d2bc1 The are some paths that were missed: -At jsonfuncs.c The errors mgs do not report about the materialize mode. -At plperl.c The function plperl_return_next_internal does not check rsi app

Re: Print warning when I execute my own extension function

2022-02-20 Thread Julien Rouhaud
Hi, On Sun, Feb 20, 2022 at 07:23:56PM +0900, Dong Wook Lee wrote: > Hi hackers, > I've read in this blog ( > http://big-elephants.com/2015-10/writing-postgres-extensions-part-i/) > and I wrote an extension about base36_encode with c code > but when I executed a query like this below I got a warni

Print warning when I execute my own extension function

2022-02-20 Thread Dong Wook Lee
Hi hackers, I've read in this blog ( http://big-elephants.com/2015-10/writing-postgres-extensions-part-i/) and I wrote an extension about base36_encode with c code but when I executed a query like this below I got a warning below. ``` postgres=# SELECT base36_encode(123); WARNING: problem in allo

Re: postgres_fdw and skip locked

2022-02-20 Thread Alexander Pyhalov
Ashutosh Bapat писал 2022-02-17 16:30: On Wed, Feb 16, 2022 at 8:38 PM Alexander Pyhalov wrote: Ashutosh Bapat писал 2022-02-16 16:40: > On Mon, Feb 14, 2022 at 4:23 PM Alexander Pyhalov > wrote: >> > I see that these options will work for all kinds of relations. So no > problem if foreign ta