Re: Add index scan progress to pg_stat_progress_vacuum

2023-01-19 Thread Masahiko Sawada
On Thu, Jan 12, 2023 at 11:02 PM Imseih (AWS), Sami wrote: > > Thanks for the feedback and I apologize for the delay in response. > > >I think the problem here is that you're basically trying to work around > > the > >lack of an asynchronous state update mechanism between leader and > >

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-19 Thread Peter Smith
On Fri, Jan 20, 2023 at 2:47 PM shveta malik wrote: > ... > 2) > Logging: > 2023-01-19 17:33:16.202 IST [404797] DEBUG: logical replication apply > delay: 19979 ms > 2023-01-19 17:33:26.212 IST [404797] DEBUG: logical replication apply > delay: 9969 ms > 2023-01-19 17:34:25.730 IST [404962]

RE: Logical replication timeout problem

2023-01-19 Thread wangw.f...@fujitsu.com
On Fri, Jan 20, 2023 at 10:10 AM Peter Smith wrote: > Here are some review comments for patch v3-0001. Thanks for your comments. > == > Commit message > > 1. > The problem is when there is a DDL in a transaction that generates lots of > temporary data due to rewrite rules, these temporary

Re: generic plans and "initial" pruning

2023-01-19 Thread Amit Langote
On Fri, Jan 20, 2023 at 12:58 PM Tom Lane wrote: > Amit Langote writes: > > On Fri, Jan 20, 2023 at 12:31 PM Tom Lane wrote: > >> It might be possible to incorporate this pointer into PlannedStmt > >> instead of passing it separately. > > > Yeah, that would be less churn. Though, I wonder if

RE: Logical replication timeout problem

2023-01-19 Thread wangw.f...@fujitsu.com
On Fri, Jan 20, 2023 at 12:35 PM Amit Kapila wrote: > On Fri, Jan 20, 2023 at 7:40 AM Peter Smith wrote: > > > > Here are some review comments for patch v3-0001. > > > > == > > src/backend/replication/logical/logical.c > > > > 3. forward declaration > > > > +/* update progress callback */ >

RE: Logical replication timeout problem

2023-01-19 Thread wangw.f...@fujitsu.com
On Thu, Jan 19, 2023 at 19:37 PM Amit Kapila wrote: > On Thu, Jan 19, 2023 at 4:13 PM Ashutosh Bapat > wrote: > > > > On Wed, Jan 18, 2023 at 6:00 PM Amit Kapila > wrote: > > > > > + */ > > > + ReorderBufferUpdateProgressCB update_progress; > > > > > > Are you suggesting changing the name of

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-19 Thread Peter Smith
Hi Osumi-san, here are my review comments for the latest patch v17-0001. == Commit Message 1. Prohibit the combination of this feature and parallel streaming mode. SUGGESTION (using the same wording as in the code comments) The combination of parallel streaming mode and min_apply_delay is

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

2023-01-19 Thread David Geier
On 1/18/23 13:52, David Geier wrote: On 1/16/23 21:39, Pavel Stehule wrote: po 16. 1. 2023 v 21:34 odesílatel Tomas Vondra napsal:     Hi,     there's minor bitrot in the Mkvcbuild.pm change, making cfbot unhappy.     As for the patch, I don't have much comments. I'm wondering if it'd

Re: Experiments with Postgres and SSL

2023-01-19 Thread Vladimir Sitnikov
>I don't think it's worth implementing a code path in > the server like this as it would then become cruft that would be hard > to ever get rid of. Do you think the server can de-support the old code path soon? > I think you can do the same thing, more or less, in the client. Like > if the

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-19 Thread Masahiko Sawada
On Thu, Jan 19, 2023 at 2:41 PM Amit Kapila wrote: > > On Wed, Jan 18, 2023 at 12:09 PM Amit Kapila wrote: > > > > On Fri, Jan 13, 2023 at 11:50 AM Peter Smith wrote: > > > > > > Here are some review comments for patch v79-0002. > > > > > > > So, this is about the latest

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-19 Thread Takamichi Osumi (Fujitsu)
Hi, Horiguchi-san and Amit-san On Wednesday, November 9, 2022 3:41 PM Kyotaro Horiguchi wrote: > Using interval is not standard as this kind of parameters but it seems > convenient. On the other hand, it's not great that the unit month introduces > some subtle ambiguity. This patch translates

Re: Stack overflow issue

2023-01-19 Thread Egor Chindyaskin
03.01.2023 22:45, Sascha Kuhl writes: Great work. Max Stack depth is memory dependent? Processor dependent? Hello! These situations are not specific to the x86_64 architecture, but also manifest themselves, for example, on aarch64 architecture. For example this query, ran on aarch64,

Re: Logical replication timeout problem

2023-01-19 Thread Peter Smith
On Fri, Jan 20, 2023 at 3:35 PM Amit Kapila wrote: > > On Fri, Jan 20, 2023 at 7:40 AM Peter Smith wrote: > > > > Here are some review comments for patch v3-0001. > > > > == > > src/backend/replication/logical/logical.c > > > > 3. forward declaration > > > > +/* update progress callback */ >

Re: Unicode grapheme clusters

2023-01-19 Thread Pavel Stehule
pá 20. 1. 2023 v 2:55 odesílatel Bruce Momjian napsal: > On Thu, Jan 19, 2023 at 07:53:43PM -0500, Tom Lane wrote: > > Bruce Momjian writes: > > > I am not sure what you are referring to above? character_length? I > was > > > talking about display length, and psql uses that --- at some point,

Re: Adjust the description of OutputPluginCallbacks in pg-doc

2023-01-19 Thread Amit Kapila
On Fri, Jan 20, 2023 at 8:03 AM wangw.f...@fujitsu.com wrote: > > On Thurs, Jan 19, 2023 at 19:18 PM Amit Kapila > wrote: > > On Wed, Jan 11, 2023 at 4:20 PM wangw.f...@fujitsu.com > > wrote: > > > > > > When I was reading the "Logical Decoding Output Plugins" chapter in pg-doc > > [1], > > >

Re: Generating code for query jumbling through gen_node_support.pl

2023-01-19 Thread Michael Paquier
On Thu, Jan 19, 2023 at 09:42:03AM +0100, Peter Eisentraut wrote: > I see that in the 0003 patch, most location fields now have an explicit > markup with query_jumble_ignore. I thought we had previously resolved to > consider location fields to be automatically ignored unless explicitly >

Re: Logical replication timeout problem

2023-01-19 Thread Amit Kapila
On Fri, Jan 20, 2023 at 7:40 AM Peter Smith wrote: > > Here are some review comments for patch v3-0001. > > == > src/backend/replication/logical/logical.c > > 3. forward declaration > > +/* update progress callback */ > +static void update_progress_cb_wrapper(ReorderBuffer *cache, > +

Re: generic plans and "initial" pruning

2023-01-19 Thread Tom Lane
Amit Langote writes: > On Fri, Jan 20, 2023 at 12:31 PM Tom Lane wrote: >> It might be possible to incorporate this pointer into PlannedStmt >> instead of passing it separately. > Yeah, that would be less churn. Though, I wonder if you still hold > that PlannedStmt should not be scribbled upon

Re: generic plans and "initial" pruning

2023-01-19 Thread Amit Langote
On Fri, Jan 20, 2023 at 12:31 PM Tom Lane wrote: > Amit Langote writes: > > On Fri, Jan 20, 2023 at 4:39 AM Tom Lane wrote: > >> I had what felt like an epiphany: the whole problem arises because the > >> system is wrongly factored. We should get rid of AcquireExecutorLocks > >> altogether,

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-19 Thread shveta malik
On Thu, Jan 19, 2023 at 12:42 PM Takamichi Osumi (Fujitsu) wrote: > > On Wednesday, January 18, 2023 4:06 PM Peter Smith > wrote: > > Here are my review comments for the latest patch v16-0001. (excluding the > > test code) > Hi, thank you for your review ! > > > == > > > > General > > > >

Re: Record queryid when auto_explain.log_verbose is on

2023-01-19 Thread Michael Paquier
On Fri, Jan 20, 2023 at 11:43:51AM +0900, torikoshia wrote: > Sorry to make you go through the trouble of looking for it. > I've now created it. > https://commitfest.postgresql.org/42/4136/ FWIW, no objections from here. This maps with EXPLAIN where the query ID is only printed under VERBOSE. --

Re: generic plans and "initial" pruning

2023-01-19 Thread Tom Lane
Amit Langote writes: > On Fri, Jan 20, 2023 at 4:39 AM Tom Lane wrote: >> I had what felt like an epiphany: the whole problem arises because the >> system is wrongly factored. We should get rid of AcquireExecutorLocks >> altogether, allowing the plancache to hand back a generic plan that >>

Re: generic plans and "initial" pruning

2023-01-19 Thread Amit Langote
On Fri, Jan 20, 2023 at 4:39 AM Tom Lane wrote: > I spent some time re-reading this whole thread, and the more I read > the less happy I got. Thanks a lot for your time on this. > We are adding a lot of complexity and introducing > coding hazards that will surely bite somebody someday. And

Re: Replace PROC_QUEUE / SHM_QUEUE with ilist.h

2023-01-19 Thread Andres Freund
Hi, On 2022-12-03 10:17:22 -0800, Andres Freund wrote: > On 2022-11-19 21:59:30 -0800, Andres Freund wrote: > > In [1] Robert justifiably complained about the use of PROC_QUEUE. I've > > previously been bothered by this in [2], but didn't get around to finishing > > the patches. > > > > One

Re: Record queryid when auto_explain.log_verbose is on

2023-01-19 Thread torikoshia
On 2023-01-19 19:05, Julien Rouhaud wrote: Hi, On Tue, Jan 17, 2023 at 10:53:23PM +0900, torikoshia wrote: > > For interactive EXPLAIN the query identifier is printed just after the > plan, > before the triggers and the JIT summary so auto_explain should do the > same. Thanks for the comment!

RE: Adjust the description of OutputPluginCallbacks in pg-doc

2023-01-19 Thread wangw.f...@fujitsu.com
On Thurs, Jan 19, 2023 at 19:18 PM Amit Kapila wrote: > On Wed, Jan 11, 2023 at 4:20 PM wangw.f...@fujitsu.com > wrote: > > > > When I was reading the "Logical Decoding Output Plugins" chapter in pg-doc > [1], > > I think in the summary section, only the callback message_cb is not > > described

Re: [EXTERNAL] Re: [PATCH] Support using "all" for the db user in pg_ident.conf

2023-01-19 Thread Michael Paquier
On Thu, Jan 19, 2023 at 12:23:16PM +0100, Jelte Fennema wrote: > should be either: > 1. a group membership check > 2. group membership checks > > Now it's mixed singular and plural. Thanks, fixed. And now applied the last patch. -- Michael signature.asc Description: PGP signature

Re: Logical replication timeout problem

2023-01-19 Thread Peter Smith
Here are some review comments for patch v3-0001. == Commit message 1. The problem is when there is a DDL in a transaction that generates lots of temporary data due to rewrite rules, these temporary data will not be processed by the pgoutput - plugin. Therefore, the previous fix (f95d53e) for

Re: pgindent vs variable declaration across multiple lines

2023-01-19 Thread Tom Lane
Andres Freund writes: > On 2023-01-19 20:43:44 -0500, Tom Lane wrote: >> What reindent-all-branches pain? We haven't done an all-branches >> reindent in the past, even for pgindent fixes that touched far more >> code than this would (assuming that the proposed fix doesn't have >> other

Re: pgindent vs variable declaration across multiple lines

2023-01-19 Thread Andres Freund
Hi, On 2023-01-19 20:43:44 -0500, Tom Lane wrote: > Andres Freund writes: > > There's a few places in the code that try to format a variable definition > > like this > > > ReorderBufferChange *next_change = > > dlist_container(ReorderBufferChange, node, next); > > > but pgindent

Re: Unicode grapheme clusters

2023-01-19 Thread Bruce Momjian
On Thu, Jan 19, 2023 at 07:53:43PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > I am not sure what you are referring to above? character_length? I was > > talking about display length, and psql uses that --- at some point, our > > lack of support for graphemes will cause psql to not align

Re: Non-superuser subscription owners

2023-01-19 Thread Andres Freund
Hi, On 2023-01-19 17:16:20 -0800, Jeff Davis wrote: > The predefined role is probably the biggest user-facing part of the > change. Does it mean that members can create any number of any kind of > subscription? I don't think we need to support complicated restriction schemes around this now. I'm

Re: Non-superuser subscription owners

2023-01-19 Thread Andres Freund
Hi, On 2023-01-19 10:45:35 -0500, Robert Haas wrote: > On Wed, Jan 18, 2023 at 3:58 PM Mark Dilger > wrote: > > > On Jan 18, 2023, at 12:51 PM, Robert Haas wrote: > > > > > > Unless I'm missing something, it seems like this could be a quite small > > > patch. > > > > I didn't like the idea of

Re: pgindent vs variable declaration across multiple lines

2023-01-19 Thread Tom Lane
Andres Freund writes: > There's a few places in the code that try to format a variable definition > like this > ReorderBufferChange *next_change = > dlist_container(ReorderBufferChange, node, next); > but pgindent turns that into > ReorderBufferChange *next_change = >

pgindent vs variable declaration across multiple lines

2023-01-19 Thread Andres Freund
Hi, There's a few places in the code that try to format a variable definition like this ReorderBufferChange *next_change = dlist_container(ReorderBufferChange, node, next); but pgindent turns that into ReorderBufferChange *next_change = dlist_container(ReorderBufferChange,

Re: Experiments with Postgres and SSL

2023-01-19 Thread Jacob Champion
On Wed, Jan 18, 2023 at 7:16 PM Greg Stark wrote: > I had a conversation a while back with Heikki where he expressed that > it was annoying that we negotiate SSL/TLS the way we do since it > introduces an extra round trip. Aside from the performance > optimization I think accepting standard TLS

Re: Non-superuser subscription owners

2023-01-19 Thread Jeff Davis
On Thu, 2023-01-19 at 14:11 -0500, Robert Haas wrote: > I guess I'm not quite seeing it. Why can't we write a small patch to > get this working right now, probably in a few hours, and deal with > any > improvements that people want at a later time? To me, it's worrisome when there are more than a

Re: PL/Python: Fix return in the middle of PG_TRY() block.

2023-01-19 Thread Andres Freund
Hi, On 2023-01-16 10:29:03 -0500, Tom Lane wrote: > Xing Guo writes: > > Are there any unsafe codes in pltcl.c? The return statement is in the > > PG_CATCH() block, I think the exception stack has been recovered in > > PG_CATCH block so the return statement in PG_CATCH block should be ok? > >

Re: Unicode grapheme clusters

2023-01-19 Thread Tom Lane
Bruce Momjian writes: > I am not sure what you are referring to above? character_length? I was > talking about display length, and psql uses that --- at some point, our > lack of support for graphemes will cause psql to not align columns. That's going to happen regardless, as long as we can't

Re: Unicode grapheme clusters

2023-01-19 Thread Bruce Momjian
On Thu, Jan 19, 2023 at 07:37:48PM -0500, Greg Stark wrote: > This is how we've always documented it. Postgres treats code points as > "characters" not graphemes. > > You don't need to go to anything as esoteric as emojis to see this either. > Accented characters like é have no canonical forms

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-01-19 Thread David Rowley
On Tue, 10 Jan 2023 at 15:08, Andres Freund wrote: > Thanks for letting me now. Updated version attached. I'm not too sure I've qualified for giving a meaningful design review here, but I have started looking at the patches and so far only made it as far as 0006. I noted down the following

Re: [PATCH] Teach planner to further optimize sort in distinct

2023-01-19 Thread David Rowley
On Fri, 20 Jan 2023 at 08:26, Ankit Kumar Pandey wrote: > > On 19/01/23 18:49, David Rowley wrote: > > You can just switch to using that function in > > create_final_distinct_paths(). You'll need to consider if the query is > > a DISTINCT ON query and not try the unordered version of the function

Re: Unicode grapheme clusters

2023-01-19 Thread Greg Stark
This is how we've always documented it. Postgres treats code points as "characters" not graphemes. You don't need to go to anything as esoteric as emojis to see this either. Accented characters like é have no canonical forms that are multiple code points and in some character sets some accented

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-19 Thread Peter Geoghegan
On Thu, Jan 19, 2023 at 3:38 PM Andres Freund wrote: > Another version of this could be to integrate analyze.c's scan more closely > with vacuum all the time. It's a bit bonkers that we often sequentially read > blocks, evict them from shared buffers if we read them, just to then > afterwards do

Re: Use appendStringInfoSpaces more

2023-01-19 Thread David Rowley
On Fri, 20 Jan 2023 at 10:23, Peter Smith wrote: > Should the add_indent function also have a check to avoid making > unnecessary calls to appendStringInfoSpaces when the level is 0? Although I didn't opt to do that, thank you for having a look. I do think the patch is trivially simple and

Re: Experiments with Postgres and SSL

2023-01-19 Thread Greg Stark
On Thu, 19 Jan 2023 at 15:49, Vladimir Sitnikov wrote: > > What if the server that supports 'fast TLS' added an extra notification in > case client connects with a classic TLS? > Then a capable client could remember host:port and try with newer TLS appoach > the next time it connects. > > It

Re: Fix GetWALAvailability function code comments for WALAVAIL_REMOVED return value

2023-01-19 Thread Tom Lane
sirisha chamarthi writes: > On Wed, Oct 19, 2022 at 7:59 PM Kyotaro Horiguchi > wrote: >> In short, the proposed fix alone seems fine to me. If we want to show >> further details, I would add a bit as follows. >> >> | * * WALAVAIL_REMOVED means it has been removed. A replication stream on >> |

Re: Use appendStringInfoSpaces more

2023-01-19 Thread David Rowley
On Fri, 20 Jan 2023 at 10:25, Tom Lane wrote: > > Peter Smith writes: > > Should the add_indent function also have a check to avoid making > > unnecessary calls to appendStringInfoSpaces when the level is 0? > > Seems like unnecessary extra notation, seeing that appendStringInfoSpaces > will

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-19 Thread Andres Freund
Hi, On 2023-01-19 15:10:38 -0800, Peter Geoghegan wrote: > On Thu, Jan 19, 2023 at 2:54 PM Andres Freund wrote: > > Yea. Hence my musing about potentially addressing this by choosing to visit > > additional blocks during the heap scan using vacuum's block sampling logic. > > I'd rather just

Re: Add LZ4 compression in pg_dump

2023-01-19 Thread Tomas Vondra
On 1/19/23 18:55, Tomas Vondra wrote: > Hi, > > On 1/19/23 17:42, gkokola...@pm.me wrote: >> >> ... >> >> Agreed. It was initially submitted as one patch. Then it was requested to be >> split up in two parts, one to expand the use of the existing API and one to >> replace with the new interface.

Re: Use fadvise in wal replay

2023-01-19 Thread Andres Freund
Hi, On 2023-01-19 22:19:10 +0100, Tomas Vondra wrote: > So I'm a bit unsure about this patch. I doesn't seem like it can perform > better than read-ahead (although perhaps it does, on a different storage > system). I really don't see the point of the patch as-is. It's not going to help OSs

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-19 Thread Peter Geoghegan
On Thu, Jan 19, 2023 at 2:54 PM Andres Freund wrote: > Yea. Hence my musing about potentially addressing this by choosing to visit > additional blocks during the heap scan using vacuum's block sampling logic. I'd rather just invent a way for vacuumlazy.c to tell the top-level vacuum.c caller "I

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-19 Thread Andres Freund
Hi, On 2023-01-19 13:22:28 -0800, Peter Geoghegan wrote: > On Thu, Jan 19, 2023 at 12:56 PM Andres Freund wrote: > > But in contrast to dead_tuples, where I think we can just stop analyze from > > updating it unless we crashed recently, I do think we need to update > > reltuples > > in vacuum.

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-19 Thread Andres Freund
Hi, On 2023-01-19 13:36:41 -0800, Peter Geoghegan wrote: > On Thu, Jan 19, 2023 at 12:58 PM Andres Freund wrote: > > There's absolutely no guarantee that autoanalyze is triggered > > there. Particularly with repeated vacuums triggered due to an relfrozenxid > > age > > that can't be advanced

Re: DSA failed to allocate memory

2023-01-19 Thread Tom Lane
Thomas Munro writes: > Thanks for the report, and for working on the fix. Can you please > create a commitfest entry (if you haven't already)? I plan to look at > this soon, after the code freeze. Hi Thomas, are you still intending to look at this DSA bug fix? It's been sitting idle for

Re: Unicode grapheme clusters

2023-01-19 Thread Bruce Momjian
On Thu, Jan 19, 2023 at 02:44:57PM +0100, Pavel Stehule wrote: > Surely it should be fixed. Unfortunately - all the terminals that I can use > don't support it. So at this moment it may be premature to fix it, because the > visual form will still be broken. Yes, none of my terminal emulators

Re: doc: mentioned CREATE+ATTACH PARTITION as an alternative to CREATE TABLE..PARTITION OF

2023-01-19 Thread Justin Pryzby
On Thu, Jan 19, 2023 at 04:47:59PM -0500, Robert Treat wrote: > I think all of that feedback is useful, I guess the immediate question > becomes if Justin wants to try to proceed with his patch implementing > the change, or if adjusting the documentation for the current > implementation is the

Re: Extracting cross-version-upgrade knowledge from buildfarm client

2023-01-19 Thread Andrew Dunstan
On 2023-01-18 We 10:33, Tom Lane wrote: > Andrew Dunstan writes: >> fairwren and drongo are clean except for fairywren upgrading 9.6 to 11. >> This appears to be a longstanding issue that the fuzz processing was >> causing us to ignore. See for example >>

Re: doc: mentioned CREATE+ATTACH PARTITION as an alternative to CREATE TABLE..PARTITION OF

2023-01-19 Thread Robert Treat
On Wed, Jan 11, 2023 at 4:13 PM Robert Haas wrote: > On Wed, Jan 11, 2023 at 10:48 AM Robert Treat wrote: > > > @Robert: I wonder why shouldn't CREATE..PARTITION OF *also* be patched > > > to first create a table, and then attach the partition, transparently > > > doing what everyone would want,

Re: Extracting cross-version-upgrade knowledge from buildfarm client

2023-01-19 Thread Tom Lane
Andrew Dunstan writes: > See > > I tested it and it seems to be doing the right thing. Yeah, seems to do what I want. Thanks! regards, tom lane

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-19 Thread Peter Geoghegan
On Thu, Jan 19, 2023 at 12:58 PM Andres Freund wrote: > There's absolutely no guarantee that autoanalyze is triggered > there. Particularly with repeated vacuums triggered due to an relfrozenxid age > that can't be advanced that very well might not happen within days on a large > relation.

Re: Operation log for major operations

2023-01-19 Thread Ted Yu
On Thu, Jan 19, 2023 at 1:12 PM Dmitry Koval wrote: > >The patch does not apply on top of HEAD ... > > Thanks! > Here is a fixed version. > > Additional changes: > 1) get_operation_log() function doesn't create empty operation log file; > 2) removed extra unlink() call. > > -- > With best

Re: Use appendStringInfoSpaces more

2023-01-19 Thread Tom Lane
Peter Smith writes: > Should the add_indent function also have a check to avoid making > unnecessary calls to appendStringInfoSpaces when the level is 0? Seems like unnecessary extra notation, seeing that appendStringInfoSpaces will fall out quickly for a zero argument.

Re: Use appendStringInfoSpaces more

2023-01-19 Thread Peter Smith
On Thu, Jan 19, 2023 at 8:45 PM David Rowley wrote: > > In [1] I noticed a bit of a poor usage of appendStringInfoString which > just appends 4 spaces in a loop, one for each indent level of the > jsonb. It should be better just to use appendStringInfoSpaces and > just append all the spaces in

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-19 Thread Peter Geoghegan
On Thu, Jan 19, 2023 at 12:56 PM Andres Freund wrote: > > In other words, ANALYZE sometimes (but not always) produces wrong answers. > > For dead tuples, but not live tuples. > > In other words, VACUUM sometimes (but not always) produces wrong answers. > > For live tuples, but not badly so for

Re: Use fadvise in wal replay

2023-01-19 Thread Tomas Vondra
Hi, I looked at this patch today. The change is fairly simple, so I decided to do a benchmark. To prepare, I created a cluster with a 1GB database, created a backup, and ran 1h UPDATE workload with WAL archiving. Then, the actual benchmark does this: 1. restore the datadir backup 2. copy the WAL

Re: Authentication fails for md5 connections if ~/.postgresql/postgresql.{crt and key} exist

2023-01-19 Thread Israel Barth Rubio
Hello Jim, > Hi Jelte, thanks for the message. You're right, an invalid cert path > does solve the issue - I even use it for tests. Although it solves the > authentication issue it still looks in my eyes like a non intuitive > workaround/hack. Perhaps a new sslmode isn't the right place for this

Re: Operation log for major operations

2023-01-19 Thread Dmitry Koval
>The patch does not apply on top of HEAD ... Thanks! Here is a fixed version. Additional changes: 1) get_operation_log() function doesn't create empty operation log file; 2) removed extra unlink() call. -- With best regards, Dmitry Koval Postgres Professional: http://postgrespro.comFrom

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-19 Thread Andres Freund
Hi On 2023-01-18 19:26:22 -0800, Peter Geoghegan wrote: > On Wed, Jan 18, 2023 at 7:04 PM Andres Freund wrote: > > > You seem to be saying that it's a problem if we don't update reltuples > > > -- an estimate -- when less than 2% of the table is scanned by VACUUM. > > > But why? Why can't we

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-19 Thread Andres Freund
Hi, On 2023-01-19 15:12:12 -0500, Robert Haas wrote: > On Wed, Jan 18, 2023 at 1:31 PM Peter Geoghegan wrote: > > pgstat_report_analyze() will totally override the > > tabentry->dead_tuples information that drives autovacuum.c, based on > > an estimate derived from a random sample -- which seems

Re: document the need to analyze partitioned tables

2023-01-19 Thread Bruce Momjian
On Thu, Jan 19, 2023 at 01:50:05PM +0100, Laurenz Albe wrote: > On Wed, 2023-01-18 at 16:23 -0500, Bruce Momjian wrote: > > Is it possible to document when partition table statistics helps? > > I think it would be difficult to come up with an exhaustive list. I was afraid of that. I asked only

Re: Experiments with Postgres and SSL

2023-01-19 Thread Vladimir Sitnikov
It would be great if PostgreSQL supported 'start with TLS', however, how could clients activate the feature? I would like to refrain users from configuring the handshake mode, and I would like to refrain from degrading performance when a new client talks to an old database. What if the server

Re: Transparent column encryption

2023-01-19 Thread Jacob Champion
On 12/31/22 06:17, Peter Eisentraut wrote: > On 21.12.22 06:46, Peter Eisentraut wrote: >> And another update.  The main changes are that I added an 'unspecified' >> CMK algorithm, which indicates that the external KMS knows what it is >> but the database system doesn't.  This was discussed a

Re: meson oddities

2023-01-19 Thread Andres Freund
Hi, On 2023-01-19 21:37:15 +0100, Peter Eisentraut wrote: > On 11.01.23 12:05, Peter Eisentraut wrote: > > I think there is also an adjacent issue:  The subdir options may be > > absolute or relative.  So if you specify --prefix=/usr/local and > > --sysconfdir=/etc/postgresql, then > > > >    

Re: meson oddities

2023-01-19 Thread Peter Eisentraut
On 11.01.23 12:05, Peter Eisentraut wrote: I think there is also an adjacent issue:  The subdir options may be absolute or relative.  So if you specify --prefix=/usr/local and --sysconfdir=/etc/postgresql, then     config_paths_data.set_quoted('SYSCONFDIR', dir_prefix / dir_sysconf) would

Re: BF animal malleefowl reported an failure in 001_password.pl

2023-01-19 Thread Tom Lane
Thomas Munro writes: > So I think we probably need something like the attached, which I was > originally trying to avoid. Yeah, something like that. I also wonder if you don't need to think a bit harder about the ordering of the flag checks, in particular it seems like servicing reload_request

Re: almost-super-user problems that we haven't fixed yet

2023-01-19 Thread Robert Haas
On Thu, Jan 19, 2023 at 2:46 PM Nathan Bossart wrote: > > Thanks. I'd move it to the inner indentation level so it's closer to > > the test at issue. > > I meant for it to cover the call to HaveNFreeProcs() as well since the same > idea applies. I left it the same for now, but if you still think

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-19 Thread Robert Haas
On Wed, Jan 18, 2023 at 1:31 PM Peter Geoghegan wrote: > pgstat_report_analyze() will totally override the > tabentry->dead_tuples information that drives autovacuum.c, based on > an estimate derived from a random sample -- which seems to me to be an > approach that just doesn't have any sound

Re: almost-super-user problems that we haven't fixed yet

2023-01-19 Thread Nathan Bossart
On Thu, Jan 19, 2023 at 02:17:35PM -0500, Robert Haas wrote: > On Thu, Jan 19, 2023 at 12:54 PM Nathan Bossart > wrote: >> > OK. Might be worth a short comment. >> >> I added one. > > Thanks. I'd move it to the inner indentation level so it's closer to > the test at issue. I meant for it to

Re: generic plans and "initial" pruning

2023-01-19 Thread Tom Lane
I spent some time re-reading this whole thread, and the more I read the less happy I got. We are adding a lot of complexity and introducing coding hazards that will surely bite somebody someday. And after awhile I had what felt like an epiphany: the whole problem arises because the system is

Re: Extracting cross-version-upgrade knowledge from buildfarm client

2023-01-19 Thread Andrew Dunstan
On 2023-01-18 We 17:14, Tom Lane wrote: > Andrew Dunstan writes: >> I think we can do what you want but it's a bit harder than what you've >> done. If we're not going to save the current run's product then we need >> to run the upgrade test from a different directory (probably directly in >>

Re: [PATCH] Teach planner to further optimize sort in distinct

2023-01-19 Thread Ankit Kumar Pandey
On 19/01/23 18:49, David Rowley wrote: I think you should write a function like: bool pathkeys_count_contained_in_unordered(List *keys1, List *keys2, List **reorderedkeys, int *n_common) which works very similarly to pathkeys> _count_contained_in, but populates *reorderedkeys so it

Re: almost-super-user problems that we haven't fixed yet

2023-01-19 Thread Robert Haas
On Thu, Jan 19, 2023 at 12:54 PM Nathan Bossart wrote: > > OK. Might be worth a short comment. > > I added one. Thanks. I'd move it to the inner indentation level so it's closer to the test at issue. I would also suggest reordering the documentation and the postgresql.conf.sample file so that

Re: Non-superuser subscription owners

2023-01-19 Thread Robert Haas
On Thu, Jan 19, 2023 at 1:40 PM Jeff Davis wrote: > On Thu, 2023-01-19 at 10:45 -0500, Robert Haas wrote: > > I wouldn't be OK with writing our own connection string parser for > > this purpose, but using PQconninfoParse seems OK. We still have to > > embed knowledge of which connection string

Re: HOT chain validation in verify_heapam()

2023-01-19 Thread Robert Haas
On Thu, Jan 19, 2023 at 8:55 AM Aleksander Alekseev wrote: > I noticed that this patch stuck a little and decided to take another look. > > It seems to be well written, covered with tests and my understanding > is that all the previous feedback was accounted for. To your knowledge > is there

Re: add PROCESS_MAIN to VACUUM

2023-01-19 Thread Nathan Bossart
On Thu, Jan 19, 2023 at 05:28:25PM +0530, vignesh C wrote: > The patch does not apply on top of HEAD as in [1], please post a rebased > patch: rebased -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 7a7f96bf4eea5be6cc252dda6bc330e77a6a3316 Mon Sep 17 00:00:00 2001 From:

Re: Non-superuser subscription owners

2023-01-19 Thread Jeff Davis
On Thu, 2023-01-19 at 10:45 -0500, Robert Haas wrote: > I wouldn't be OK with writing our own connection string parser for > this purpose, but using PQconninfoParse seems OK. We still have to > embed knowledge of which connection string parameters can trigger > local file access, but that doesn't

Re: BF animal malleefowl reported an failure in 001_password.pl

2023-01-19 Thread Thomas Munro
On Tue, Jan 17, 2023 at 11:24 AM Thomas Munro wrote: > Another idea would be to teach the latch infrastructure itself to > magically swap latch events to position 0. Latches are usually > prioritised; it's only in this rare race case that they are not. I liked that idea for a while, but I

Re: Non-superuser subscription owners

2023-01-19 Thread Jeff Davis
On Wed, 2023-01-18 at 14:38 -0500, Robert Haas wrote: > I was just noticing that what was committed here didn't actually fix > the problem implied by the subject line. That is, non-superuser still > can't own subscriptions. To put that another way, there's no way for > the superuser to delegate

Re: Remove source code display from \df+?

2023-01-19 Thread Isaac Morland
On Thu, 19 Jan 2023 at 11:30, Justin Pryzby wrote: > On Wed, Jan 18, 2023 at 10:27:46AM -0500, Isaac Morland wrote: > > > > I thought I had: https://commitfest.postgresql.org/42/4133/ > > This is failing tests: > http://cfbot.cputube.org/isaac-morland.html > > It seems like any "make check"

Re: Doc: Rework contrib appendix -- informative titles, tweaked sentences

2023-01-19 Thread Karl O. Pinc
On Thu, 19 Jan 2023 11:03:53 -0600 "Karl O. Pinc" wrote: > Attached are 2 patches, a regular and a delta from your v4 review: > > contrib_v5-delta.patch.txt > contrib_v5.patch.txt I left your appendix title unchanged: "Additional Supplied Extensions and Modules". I had put "Extensions"

Re: Add LZ4 compression in pg_dump

2023-01-19 Thread Tomas Vondra
Hi, On 1/19/23 17:42, gkokola...@pm.me wrote: > > --- Original Message --- > On Thursday, January 19th, 2023 at 4:45 PM, Tomas Vondra > wrote: >> >> On 1/18/23 20:05, gkokola...@pm.me wrote: >> >>> --- Original Message --- >>> On Wednesday, January 18th, 2023 at 3:00 PM, Tomas

Re: Support logical replication of DDLs

2023-01-19 Thread Zheng Li
On Thu, Jan 19, 2023 at 2:05 AM Amit Kapila wrote: > > On Thu, Jan 19, 2023 at 8:39 AM Zheng Li wrote: > > > > On Wed, Jan 18, 2023 at 6:27 AM Amit Kapila wrote: > > > > > > On Sat, Jan 7, 2023 at 8:58 PM Zheng Li wrote: > > > > > > > > Foreign Tables can also be considered replicated with DDL

Re: almost-super-user problems that we haven't fixed yet

2023-01-19 Thread Nathan Bossart
On Thu, Jan 19, 2023 at 11:40:53AM -0500, Robert Haas wrote: > On Wed, Jan 18, 2023 at 4:14 PM Nathan Bossart > wrote: >> On Wed, Jan 18, 2023 at 02:51:38PM -0500, Robert Haas wrote: >> > Should (nfree < SuperuserReservedBackends) be using <=, or am I confused? >> >> I believe < is correct. At

Re: Add semi-join pushdown to postgres_fdw

2023-01-19 Thread Tomas Vondra
Hi. I took a quick look at the patch. It needs a rebase, although it applies fine using patch. A couple minor comments: 1) addl_conds seems a bit hard to understand, I'd use either the full wording (additional_conds) or maybe extra_conds 2) some of the lines got quite long, and need a wrap 3)

Re: minor bug

2023-01-19 Thread Tom Lane
=?UTF-8?Q?Torsten_F=C3=B6rtsch?= writes: > Why not > (void)getRecordTimestamp(record, ); > if (recoveryTarget == RECOVERY_TARGET_TIME) > ... Could've done it like that, but I already pushed the other version, and I don't think it's worth the trouble to change. regards,

Re: minor bug

2023-01-19 Thread Tom Lane
Laurenz Albe writes: > On Wed, 2023-01-18 at 15:03 -0500, Tom Lane wrote: >> Ah, but that only happens if recoveryTarget == RECOVERY_TARGET_TIME. >> Digging in the git history, I see that this did use to work as >> I remember: we always extracted the record time before printing it. >> That was

Re: minor bug

2023-01-19 Thread Torsten Förtsch
If we never expect getRecordTimestamp to fail, then why put it in the if-condition? getRecordTimestamp can fail if the record is not a restore point nor a commit or abort record. A few lines before in the same function there is this: /* Otherwise we only consider stopping before COMMIT or ABORT

Re: vac_update_datfrozenxid will raise "wrong tuple length" if pg_database tuple contains toast attribute.

2023-01-19 Thread Arthur Nascimento
On Thu, 19 Jan 2023 at 14:10, Nathan Bossart wrote: > This was recently back-patched [0] [1] [2]. Oh, I see that now. Thanks! Sorry for the noise. -- Arthur Nascimento EDB

Re: vac_update_datfrozenxid will raise "wrong tuple length" if pg_database tuple contains toast attribute.

2023-01-19 Thread Nathan Bossart
On Thu, Jan 19, 2023 at 10:42:47AM -0300, Arthur Nascimento wrote: > Would it be possible to backport the patch to 12 and 13? This was recently back-patched [0] [1] [2]. [0] https://postgr.es/m/y70xnvbuwqsr2...@paquier.xyz [1]

  1   2   >