Re: Add index scan progress to pg_stat_progress_vacuum

2022-11-07 Thread Masahiko Sawada
On Thu, Nov 3, 2022 at 1:52 AM Imseih (AWS), Sami wrote: > > Attached is v13-0001--Show-progress-for-index-vacuums.patch which addresses > the latest comments. Thank you for updating the patch! > 4/ Went back to calling parallel_vacuum_progress_report inside > WaitForParallelWorkersToFinish to

Re: Suppressing useless wakeups in walreceiver

2022-11-07 Thread Thomas Munro
On Tue, Nov 8, 2022 at 3:20 PM Thomas Munro wrote: > On Sun, Nov 6, 2022 at 12:01 PM Nathan Bossart > wrote: > > Here is a new version of the patch that addresses this feedback. > > This looks pretty good to me. Thanks for picking it up! I can live > with the change to use a global variable;

Re: Reviving lost replication slots

2022-11-07 Thread Bharath Rupireddy
On Tue, Nov 8, 2022 at 12:08 PM sirisha chamarthi wrote: > > On Fri, Nov 4, 2022 at 11:02 PM Amit Kapila wrote: >> >> On Fri, Nov 4, 2022 at 1:40 PM sirisha chamarthi >> wrote: >> > >> > A replication slot can be lost when a subscriber is not able to catch up >> > with the load on the primary

Re: Use proc instead of MyProc in ProcArrayGroupClearXid()/TransactionGroupUpdateXidStatus()

2022-11-07 Thread rajesh singarapu
Thanks Bharat and Amit for the review and explaining rationale. for the TransactionGroupUpdateXidStatus() change, let me see if I can piggy back this change on something more valuable. thanks Rajesh On Tue, Nov 8, 2022 at 11:58 AM Amit Kapila wrote: > > On Mon, Nov 7, 2022 at 3:17 PM rajesh

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-11-07 Thread Richard Guo
On Tue, Nov 8, 2022 at 9:31 AM David Rowley wrote: > I've been playing around with the attached patch which does: > > 1. Adjusts add_paths_to_grouping_rel so that we don't add a Sort path > when we can add an Incremental Sort path instead. This removes quite a > few redundant lines of code.

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-11-07 Thread Pavel Luzanov
On 08.11.2022 04:31, David Rowley wrote: I've been playing around with the attached patch which does: 1. Adjusts add_paths_to_grouping_rel so that we don't add a Sort path when we can add an Incremental Sort path instead. This removes quite a few redundant lines of code. 2. Removes the * 1.5

Re: [patch] Have psql's \d+ indicate foreign partitions

2022-11-07 Thread Ian Lawrence Barwick
2022年11月8日(火) 14:49 Michael Paquier : > > On Mon, Nov 07, 2022 at 01:43:22AM -0500, Tom Lane wrote: > > WFM. > > Okay, applied as bd95816, then. Thanks! CF entry updated accordingly. Regards Ian Barwick

Re: Reviving lost replication slots

2022-11-07 Thread sirisha chamarthi
Hi Amit, Thanks for your comments! On Fri, Nov 4, 2022 at 11:02 PM Amit Kapila wrote: > On Fri, Nov 4, 2022 at 1:40 PM sirisha chamarthi > wrote: > > > > A replication slot can be lost when a subscriber is not able to catch up > with the load on the primary and the WAL to catch up exceeds >

Re: Use proc instead of MyProc in ProcArrayGroupClearXid()/TransactionGroupUpdateXidStatus()

2022-11-07 Thread Amit Kapila
On Mon, Nov 7, 2022 at 3:17 PM rajesh singarapu wrote: > > In both TransactionGroupUpdateXidStatus and ProcArrayGroupClearXid > global MyProc is used. for consistency, replaced with a function local > variable. > In ProcArrayGroupClearXid(), currently, we always pass MyProc as proc, so the

Re: Use proc instead of MyProc in ProcArrayGroupClearXid()/TransactionGroupUpdateXidStatus()

2022-11-07 Thread Bharath Rupireddy
On Mon, Nov 7, 2022 at 3:17 PM rajesh singarapu wrote: > > Hi, > > In both TransactionGroupUpdateXidStatus and ProcArrayGroupClearXid > global MyProc is used. for consistency, replaced with a function local > variable. if (nextproc != MyProc)

Re: generic plans and "initial" pruning

2022-11-07 Thread Amit Langote
On Thu, Oct 27, 2022 at 11:41 AM Amit Langote wrote: > On Mon, Oct 17, 2022 at 6:29 PM Amit Langote wrote: > > On Wed, Oct 12, 2022 at 4:36 PM Amit Langote > > wrote: > > > On Fri, Jul 29, 2022 at 1:20 PM Amit Langote > > > wrote: > > > > On Thu, Jul 28, 2022 at 1:27 AM Robert Haas > > > >

Re: [patch] Have psql's \d+ indicate foreign partitions

2022-11-07 Thread Michael Paquier
On Mon, Nov 07, 2022 at 01:43:22AM -0500, Tom Lane wrote: > WFM. Okay, applied as bd95816, then. -- Michael signature.asc Description: PGP signature

Re: allow segment size to be set to < 1GiB

2022-11-07 Thread Bharath Rupireddy
On Tue, Nov 8, 2022 at 8:06 AM Tom Lane wrote: > > Andres Freund writes: > > On 2022-11-07 12:52:25 -0500, Tom Lane wrote: > >> How about instead allowing the segment size to be set in pages? > > > In addition or instead of --with-segsize/-Dsegsize? > > In addition to. What I meant by "instead"

RE: logical replication restrictions

2022-11-07 Thread Hayato Kuroda (Fujitsu)
Dear Euler, Do you have enough time to handle the issue? Our discussion has been suspended for two months... If you could not allocate a time to discuss this problem because of other important tasks or events, we would like to take over the thread and modify your patch. We've planned that we

Re: psql: Add command to use extended query protocol

2022-11-07 Thread Pavel Stehule
út 8. 11. 2022 v 5:21 odesílatel David G. Johnston < david.g.johns...@gmail.com> napsal: > On Mon, Nov 7, 2022 at 9:02 PM Pavel Stehule > wrote: > >> >> >> út 8. 11. 2022 v 3:47 odesílatel Corey Huinker >> napsal: >> >>> On Mon, Nov 7, 2022 at 4:12 PM Tom Lane wrote: >>> Corey Huinker

Re: psql: Add command to use extended query protocol

2022-11-07 Thread David G. Johnston
On Mon, Nov 7, 2022 at 9:02 PM Pavel Stehule wrote: > > > út 8. 11. 2022 v 3:47 odesílatel Corey Huinker > napsal: > >> On Mon, Nov 7, 2022 at 4:12 PM Tom Lane wrote: >> >>> Corey Huinker writes: >>> > I thought about basically reserving the \$[0-9]+ space as bind >>> variables, >>> > but it

RE: Data is copied twice when specifying both child and parent table in publication

2022-11-07 Thread Takamichi Osumi (Fujitsu)
On Monday, October 17, 2022 2:49 PM Wang, Wei/王 威 wrote: > Attach the new patch set. Hi, thank you for posting the new patches. Here are minor comments on the HEAD_v13-0002. (1) Suggestion for the document description + + If a root partitioned table is published by any

Re: Tracking last scan time

2022-11-07 Thread Michael Paquier
On Mon, Nov 07, 2022 at 04:54:07PM +0900, Michael Paquier wrote: > FWIW, all the other areas of pgstatfuncs.c manipulate timestamptz > fields with a style like the attached. That's a nit, still per the > role of consistency with the surroundings.. > > Anyway, it seems to me that a regression

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2022-11-07 Thread Michael Paquier
On Tue, Nov 01, 2022 at 08:32:48AM +0530, Bharath Rupireddy wrote: > I'm attaching the v9 patch from upthread here again for further review > and to make CF bot happy. So, I have looked at that, and at the end concluded that Andres' suggestion to use PGAlignedBlock in pg_write_zeros() will serve

Re: psql: Add command to use extended query protocol

2022-11-07 Thread Pavel Stehule
út 8. 11. 2022 v 3:47 odesílatel Corey Huinker napsal: > On Mon, Nov 7, 2022 at 4:12 PM Tom Lane wrote: > >> Corey Huinker writes: >> > I thought about basically reserving the \$[0-9]+ space as bind >> variables, >> > but it is possible, though unlikely, that users have been naming their >> >

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread David Rowley
On Tue, 8 Nov 2022 at 15:17, John Naylor wrote: > > > On Tue, Nov 8, 2022 at 8:57 AM David Rowley wrote: > > Is there anything we could align to CPU cacheline size that would > > speed something up? > > InitCatCache() already has this, which could benefit from simpler notation. Thanks. I wasn't

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread David Rowley
On Tue, 8 Nov 2022 at 14:57, David Rowley wrote: > On Tue, 8 Nov 2022 at 05:24, Andres Freund wrote: > > Should we handle the case where we get a suitably aligned pointer from > > MemoryContextAllocExtended() differently? > > Maybe it would be worth the extra check. I'm trying to imagine future

RE: Perform streaming logical transactions by background workers and parallel apply

2022-11-07 Thread Hayato Kuroda (Fujitsu)
> Fair point. I think if the user wants, she can join with > pg_stat_subscription based on PID and find the corresponding > subscription. However, if we want to identify everything via pg_locks > then I think we should also mention classid or database id as field1. > So, it would look like:

Re: New docs chapter on Transaction Management and related changes

2022-11-07 Thread Bruce Momjian
On Tue, Nov 8, 2022 at 04:37:59AM +0100, Laurenz Albe wrote: > On Mon, 2022-11-07 at 23:04 +0100, Laurenz Albe wrote: > > On Sat, 2022-11-05 at 10:08 +, Simon Riggs wrote: > > > Agreed; new compilation patch attached, including mine and then > > > Robert's suggested rewordings. > > > >

Re: New docs chapter on Transaction Management and related changes

2022-11-07 Thread Bruce Momjian
On Mon, Nov 7, 2022 at 10:58:05AM +, Simon Riggs wrote: > What I've posted is the merged patch, i.e. your latest patch, plus > changes to RELEASE SAVEPOINT from you on Oct 16, plus changes based on > the later comments from Robert and I. Thanks. I have two changes to your patch. First, I

Re: New docs chapter on Transaction Management and related changes

2022-11-07 Thread Laurenz Albe
On Mon, 2022-11-07 at 23:04 +0100, Laurenz Albe wrote: > On Sat, 2022-11-05 at 10:08 +, Simon Riggs wrote: > > Agreed; new compilation patch attached, including mine and then > > Robert's suggested rewordings. > > Thanks.  There is clearly a lot of usefule information in this. > > Some

Re: Allow single table VACUUM in transaction block

2022-11-07 Thread Simon Riggs
On Mon, 7 Nov 2022 at 08:20, Simon Riggs wrote: > Temp tables are actually easier, since we don't need any of the > concurrency features we get with lazy vacuum. So the answer is to > always run a VACUUM FULL on temp tables since this skips any issues > with indexes etc.. So I see 3 options for

RE: Logical replication timeout problem

2022-11-07 Thread wangw.f...@fujitsu.com
On Fri, Nov 4, 2022 at 18:13 PM Fabrice Chapuis wrote: > Hello Wang, > > I tested the draft patch in my lab for Postgres 14.4, the refresh of the > materialized view ran without generating the timeout on the worker. > Do you plan to propose this patch at the next commit fest. Thanks for your

Re: psql: Add command to use extended query protocol

2022-11-07 Thread Corey Huinker
On Mon, Nov 7, 2022 at 4:12 PM Tom Lane wrote: > Corey Huinker writes: > > I thought about basically reserving the \$[0-9]+ space as bind variables, > > but it is possible, though unlikely, that users have been naming their > > variables like that. > > Don't we already reserve that syntax as

Re: allow segment size to be set to < 1GiB

2022-11-07 Thread Tom Lane
Andres Freund writes: > On 2022-11-07 12:52:25 -0500, Tom Lane wrote: >> How about instead allowing the segment size to be set in pages? > In addition or instead of --with-segsize/-Dsegsize? In addition to. What I meant by "instead" was to replace your proposal of --with-segsize-mb. > Just

Re: In-placre persistance change of a relation

2022-11-07 Thread Kyotaro Horiguchi
At Fri, 4 Nov 2022 09:32:52 +0900, Ian Lawrence Barwick wrote in > cfbot reports the patch no longer applies. As CommitFest 2022-11 is > currently underway, this would be an excellent time to update the patch. Indeed, thanks! I'll do that in a few days. regards. -- Kyotaro Horiguchi NTT

Re: allow segment size to be set to < 1GiB

2022-11-07 Thread Andres Freund
Hi, On 2022-11-07 12:52:25 -0500, Tom Lane wrote: > Andres Freund writes: > > In the attached patch I renamed --with-segsize= to --with-segsize-mb= / > > -Dsegsize= to -Dsegsize_mb=, to avoid somebody building with > > --with-segsize=2 > > or such suddenly ending up with an incompatible build.

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread Andres Freund
Hi, On 2022-11-08 15:01:18 +1300, David Rowley wrote: > Andres has suggested I remove the repalloc stuff I added but see my reply to > that. I'm fine with keeping it, I just couldn't really think of cases that have strict alignment requirements but also requires resizing. Greetings, Andres

Re: Suppressing useless wakeups in walreceiver

2022-11-07 Thread Thomas Munro
On Sun, Nov 6, 2022 at 12:01 PM Nathan Bossart wrote: > Here is a new version of the patch that addresses this feedback. This looks pretty good to me. Thanks for picking it up! I can live with the change to use a global variable; it seems we couldn't quite decide what the scope was for moving

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread John Naylor
On Tue, Nov 8, 2022 at 8:57 AM David Rowley wrote: > Is there anything we could align to CPU cacheline size that would > speed something up? InitCatCache() already has this, which could benefit from simpler notation. /* * Allocate a new cache structure, aligning to a cacheline boundary * *

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread David Rowley
On Fri, 4 Nov 2022 at 04:20, Maxim Orlov wrote: > I've done a quick look and the patch is looks good to me. > Let's add tests for these functions, should we? If you think this is an > overkill, feel free to trim tests for your taste. Thanks for doing that. I'm keen to wait a bit and see if we

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread David Rowley
Thanks for having a look. On Tue, 8 Nov 2022 at 05:24, Andres Freund wrote: > FWIW, to me this is really more part of mcxt.c than its own > allocator... Particularly because MemoryContextAllocAligned() et al are > implemented there. I'm on the fence about this one. I thought it was nice that we

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-11-07 Thread David Rowley
On Tue, 8 Nov 2022 at 03:53, Ronan Dunklau wrote: > I can't see why an incrementalsort could be more expensive than a full sort, > using the same presorted path. It looks to me that in that case we should > always prefer an incrementalsort. Maybe we should bound incremental sorts cost > to make

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2022-11-07 Thread Jacob Champion
On Thu, Nov 3, 2022 at 4:39 PM Jacob Champion wrote: > There is an additional test failure with LibreSSL, which doesn't appear > to honor the SSL_CERT_FILE environment variable. This isn't a problem in > production -- if you're using LibreSSL, you'd presumably understand that > you can't use that

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-07 Thread Michael Paquier
On Mon, Nov 07, 2022 at 03:07:15PM +0900, Michael Paquier wrote: > Attached is a set of three patches: > - 0001 changes tokenize_inc_file() to use AbsoluteConfigLocation(). > AbsoluteConfigLocation() uses a static buffer and a MAXPGPATH, but > we'd rather change it to use a palloc()+strcpy()

Re: Allow single table VACUUM in transaction block

2022-11-07 Thread Peter Geoghegan
On Mon, Nov 7, 2022 at 12:20 AM Simon Riggs wrote: > > Another related idea: better behavior in the event of a manually > > issued VACUUM (now just an enqueued autovacuum) that cannot do useful > > work due to the presence of a long running snapshot. The VACUUM > > doesn't have to dutifully

Re: new option to allow pg_rewind to run without full_page_writes

2022-11-07 Thread Andres Freund
Hi, On 2022-11-08 00:07:09 +0100, Jérémie Grauer wrote: > On 06/11/2022 03:38, Andres Freund wrote: > > Hi, > > > > On 2022-11-03 16:54:13 +0100, Jérémie Grauer wrote: > > > Currently pg_rewind refuses to run if full_page_writes is off. This is to > > > prevent it to run into a torn page during

Re: [DOCS] Stats views and functions not in order?

2022-11-07 Thread Peter Smith
On Mon, Nov 7, 2022 at 5:50 AM Tom Lane wrote: > > Peter Smith writes: > > Sorry, I forgot the attachments in the previous post. PSA. > > I spent a bit of time looking at this. I agree that a lot of the > current ordering choices here look like they were made with the > advice of a dartboard,

Re: new option to allow pg_rewind to run without full_page_writes

2022-11-07 Thread Thomas Munro
On Tue, Nov 8, 2022 at 12:07 PM Jérémie Grauer wrote: > On 06/11/2022 03:38, Andres Freund wrote: > > On 2022-11-03 16:54:13 +0100, Jérémie Grauer wrote: > >> Currently pg_rewind refuses to run if full_page_writes is off. This is to > >> prevent it to run into a torn page during operation. > >> >

Re: new option to allow pg_rewind to run without full_page_writes

2022-11-07 Thread Jérémie Grauer
Hello, First, thank you for reviewing. ZFS writes files in increment of its configured recordsize for the current filesystem dataset. So with a recordsize configured to be a multiple of 8K, you can't get torn pages on writes, that's why full_page_writes can be safely deactivated on ZFS

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-07 Thread David Christensen
Hi Justin et al, Enclosed is v5 of this patch which now passes the CirrusCI checks for all supported OSes. I went ahead and reworked the test a bit so it's a little more amenable to the OS-agnostic approach for testing. Best, David

Re: Large Pages and Super Pages for PostgreSQL

2022-11-07 Thread Thomas Munro
On Sun, Jan 16, 2022 at 8:32 PM Thomas Munro wrote: > On Sun, Jan 16, 2022 at 6:03 PM DEVOPS_WwIT wrote: > > Solaris and FreeBSD supports large/super pages, and can be used > > automatically by applications. > > > > Seems Postgres can't use the large/super pages on Solaris and FreeBSD > > os(I

Re: Improve logging when using Huge Pages

2022-11-07 Thread Thomas Munro
On Tue, Nov 8, 2022 at 4:56 AM Tom Lane wrote: > Andres Freund writes: > > On 2022-11-06 14:04:29 +0700, John Naylor wrote: > >> I think the best thing to do is change huge_pages='on' to log a WARNING and > >> fallback to regular pages if the mapping fails. > > > I am strongly opposed to making

Re: New docs chapter on Transaction Management and related changes

2022-11-07 Thread Laurenz Albe
On Sat, 2022-11-05 at 10:08 +, Simon Riggs wrote: > Agreed; new compilation patch attached, including mine and then > Robert's suggested rewordings. Thanks. There is clearly a lot of usefule information in this. Some comments: > --- a/doc/src/sgml/func.sgml > +++ b/doc/src/sgml/func.sgml >

Re: PL/pgSQL cursors should get generated portal names by default

2022-11-07 Thread Jan Wieck
My comments were in no way meant as an argument for or against the change itself. Only to clearly document the side effect it will have. Regards, Jan On 11/7/22 11:57, Kirk Wolak wrote: On Mon, Nov 7, 2022 at 11:10 AM Jan Wieck > wrote: On 11/4/22 19:46, Tom

Re: thinko in basic_archive.c

2022-11-07 Thread Nathan Bossart
On Mon, Nov 07, 2022 at 04:53:35PM +0530, Bharath Rupireddy wrote: > The tempfile name can vary in size for the simple reason that a pid > can be of varying digits - for instance, tempfile name is 'foo1234' > (pid being 1234) and it becomes '\'\0\'oo1234' if we just reset the > first char to '\0'

Re: Add tracking of backend memory allocated to pg_stat_activity

2022-11-07 Thread Melanie Plageman
On Fri, Nov 04, 2022 at 08:56:13AM -0400, Reid Thompson wrote: > From a8de5d29c0c6f10962181926a49ad4fec1e52bd1 Mon Sep 17 00:00:00 2001 > From: Reid Thompson > Date: Thu, 11 Aug 2022 12:01:25 -0400 > Subject: [PATCH] Add tracking of backend memory allocated to pg_stat_activity > > This new field

Re: archive modules

2022-11-07 Thread Nathan Bossart
On Mon, Nov 07, 2022 at 03:20:31PM +0900, Michael Paquier wrote: > On Sat, Nov 05, 2022 at 02:08:58PM -0700, Nathan Bossart wrote: >> Perhaps we could eventually move the archive_command functionality to a >> contrib module (i.e., "shell_archive") so that users must always set >> archive_library.

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-07 Thread Simon Riggs
On Tue, 1 Nov 2022 at 08:55, Julien Tachoires wrote: > > > > 4. Test results with transaction 1 > > > > > > TPS vs number of sub-transaction > > > > > > nsubx HEAD patched > > > > > > 0 441109 439474 > > > 8 439045 438103 > > > 16 439123 436993 > > > 24

Re: psql: Add command to use extended query protocol

2022-11-07 Thread Tom Lane
Corey Huinker writes: > I thought about basically reserving the \$[0-9]+ space as bind variables, > but it is possible, though unlikely, that users have been naming their > variables like that. Don't we already reserve that syntax as Params? Not sure whether there would be any conflicts versus

Re: [PATCH] pgbench: add multiconnect option

2022-11-07 Thread Fabien COELHO
Hello Ian, cfbot reports the patch no longer applies. As CommitFest 2022-11 is currently underway, this would be an excellent time to update the patch. Attached a v5 which is just a rebase. -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-11-07 Thread Pavel Luzanov
On 07.11.2022 20:30, Ronan Dunklau wrote: What I meant here is that disabling seqscans, the planner still chooses a full sort over a partial sort. The underlying index is the same, it is just a matter of choosing a Sort node over an IncrementalSort node. This, I think, is wrong: I can't see how

Re: psql: Add command to use extended query protocol

2022-11-07 Thread Corey Huinker
> > > > what about introduction new syntax for psql variables that should be > passed as bind variables. > I thought about basically reserving the \$[0-9]+ space as bind variables, but it is possible, though unlikely, that users have been naming their variables like that. It's unclear from your

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2022-11-07 Thread Maciek Sakrejda
On Thu, Nov 3, 2022 at 10:00 AM Melanie Plageman wrote: > > I decided not to call it pg_statio because all of the other stats views > have an underscore after stat and I thought it was an opportunity to be > consistent with them. Oh, got it. Makes sense. > > I'm reviewing the rendered docs now,

Re: allow segment size to be set to < 1GiB

2022-11-07 Thread Tom Lane
Andres Freund writes: > In the attached patch I renamed --with-segsize= to --with-segsize-mb= / > -Dsegsize= to -Dsegsize_mb=, to avoid somebody building with --with-segsize=2 > or such suddenly ending up with an incompatible build. For the purpose of exercising these code paths with the

Re: hash_xlog_split_allocate_page: failed to acquire cleanup lock

2022-11-07 Thread Robert Haas
On Mon, Oct 31, 2022 at 11:49 PM Amit Kapila wrote: > I am fine with any of those. Would you like to commit or do you prefer > me to take care of this? Sorry for not responding to this sooner. I think it's too late to do anything about this for the current round of releases at this point, but I

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-11-07 Thread Ronan Dunklau
Le lundi 7 novembre 2022, 17:58:50 CET Pavel Luzanov a écrit : > > I can't see why an incrementalsort could be more expensive than a full > > sort, using the same presorted path. > > The only reason I can see is the number of buffers to read. In the plan > with incremental sort we read the whole

allow segment size to be set to < 1GiB

2022-11-07 Thread Andres Freund
Hi, I was recently reminded of my previous desire to allow setting the segment size to less than 1GB. It's pretty painful to test large amount of segments with a segment size of 1GB, certainly our regression test don't cover anything with multiple segments. This likely wouldn't have detected the

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-11-07 Thread Pavel Luzanov
Hi, On 07.11.2022 17:53, Ronan Dunklau wrote: In your exact use case, the combo incremental-sort + Index scan is evaluated to cost more than doing a full sort + seqscan. I think we can trace that back to incremental sort being pessimistic about it's performance. If you try the same query,

Re: PL/pgSQL cursors should get generated portal names by default

2022-11-07 Thread Pavel Stehule
Dne po 7. 11. 2022 17:10 uživatel Jan Wieck napsal: > On 11/4/22 19:46, Tom Lane wrote: > > Jan Wieck writes: > >> I need to do some testing on this. I seem to recall that the naming was > >> originally done because a reference cursor is basically a named cursor > >> that can be handed around

Re: 2022-11-10 release announcement draft

2022-11-07 Thread Jonathan S. Katz
On 11/7/22 10:59 AM, Erikjan Rijkers wrote: Op 07-11-2022 om 16:51 schreef Jonathan S. Katz: Hi, Attached is a draft of the release announcement for the 2022-11-10 release. Please provide feedback no later than 2022-11-10 0:00 AoE[1]. 'now exists'  should be (I think) 'now exits'

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread Andres Freund
Hi, On 2022-11-02 00:28:46 +1300, David Rowley wrote: > Part of the work that Thomas mentions in [1], regarding Direct I/O, > diff --git a/src/backend/utils/mmgr/alignedalloc.c > b/src/backend/utils/mmgr/alignedalloc.c > new file mode 100644 > index 00..e581772758 > --- /dev/null > +++

Re: PL/pgSQL cursors should get generated portal names by default

2022-11-07 Thread Jan Wieck
On 11/4/22 19:46, Tom Lane wrote: Jan Wieck writes: I need to do some testing on this. I seem to recall that the naming was originally done because a reference cursor is basically a named cursor that can be handed around between functions and even the top SQL level of the application. For

Re: 2022-11-10 release announcement draft

2022-11-07 Thread Erikjan Rijkers
Op 07-11-2022 om 16:51 schreef Jonathan S. Katz: Hi, Attached is a draft of the release announcement for the 2022-11-10 release. Please provide feedback no later than 2022-11-10 0:00 AoE[1]. 'now exists' should be (I think) 'now exits' Erik Thanks, Jonathan [1]

Re: Improve logging when using Huge Pages

2022-11-07 Thread Tom Lane
Andres Freund writes: > On 2022-11-06 14:04:29 +0700, John Naylor wrote: >> I think the best thing to do is change huge_pages='on' to log a WARNING and >> fallback to regular pages if the mapping fails. > I am strongly opposed to making 'on' fall back to not using huge pages. That's > what 'try'

2022-11-10 release announcement draft

2022-11-07 Thread Jonathan S. Katz
Hi, Attached is a draft of the release announcement for the 2022-11-10 release. Please provide feedback no later than 2022-11-10 0:00 AoE[1]. Thanks, Jonathan [1] https://en.wikipedia.org/wiki/Anywhere_on_Earth The PostgreSQL Global Development Group has released an update to all supported

Re: Draft back-branch release notes are up

2022-11-07 Thread Jonathan S. Katz
On 11/7/22 10:30 AM, Jonathan S. Katz wrote: On 11/6/22 11:14 AM, Tom Lane wrote: Justin Pryzby writes: +  Fix planner failure with extended statistics on partitioned tables +  (Richard Guo, Justin Pryzby) This can also happen with inheritance tables. +  Add missing guards

Re: Draft back-branch release notes are up

2022-11-07 Thread Jonathan S. Katz
On 11/6/22 11:14 AM, Tom Lane wrote: Justin Pryzby writes: + Fix planner failure with extended statistics on partitioned tables + (Richard Guo, Justin Pryzby) This can also happen with inheritance tables. + Add missing guards for NULL connection pointer Maybe should be

Re: Improve logging when using Huge Pages

2022-11-07 Thread Andres Freund
Hi, On 2022-11-06 14:04:29 +0700, John Naylor wrote: > I think the best thing to do is change huge_pages='on' to log a WARNING and > fallback to regular pages if the mapping fails. That way, both dev and prod > can keep the same settings, since 'on' will have both visibility and > robustness. I

Re: WIN32 pg_import_system_collations

2022-11-07 Thread Peter Eisentraut
On 04.11.22 23:08, Juan José Santamaría Flecha wrote: Ok, I can definitely improve the comments for that function. Also consider describing in the commit message what you are doing in more detail, including some of the things that have been discussed in this thread. Going through

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-11-07 Thread Ronan Dunklau
Le samedi 5 novembre 2022, 09:51:23 CET Pavel Luzanov a écrit : > While playing with the patch I found a situation where the performance > may be degraded compared to previous versions. > > The test case below. > If you create a proper index for the query (a,c), version 16 wins. On my > notebook,

Re: Postgres auto vacuum - Disable

2022-11-07 Thread Karthik Jagadish (kjagadis)
Hi Again, Is there any difference in the way vacuum is handled in postgres9.6 and postgres12.9, We are noticing the below issue of waiting process only after upgrading to postgres12.5 $ ps -ef | grep 'waiting' postgres 8833 62646 0 Jul28 ?00:00:00 postgres: postgres cgms [local]

Re: Code checks for App Devs, using new options for transaction behavior

2022-11-07 Thread Simon Riggs
On Wed, 2 Nov 2022 at 07:40, Simon Riggs wrote: > > What will be the behavior if someone declares a savepoint with this > > name ("_internal_nested_xact"). Will this interfere with this new > > functionality? > > Clearly! I guess you are saying we should disallow them. > > > Have we tested

Re: Skipping schema changes in publication

2022-11-07 Thread vignesh C
On Fri, 4 Nov 2022 at 08:19, Ian Lawrence Barwick wrote: > > Hi > > cfbot reports the patch no longer applies [1]. As CommitFest 2022-11 is > currently underway, this would be an excellent time to update the patch. > > [1] http://cfbot.cputube.org/patch_40_3646.log Here is an updated patch which

Re: Postgres auto vacuum - Disable

2022-11-07 Thread Julien Rouhaud
Hi, On Mon, Nov 07, 2022 at 02:22:56PM +0100, Laurenz Albe wrote: > On Mon, 2022-11-07 at 12:12 +, Karthik Jagadish (kjagadis) wrote: > > I have follow-up question where the vacuum process is waiting and not doing > > it’s job. > > When we grep on waiting process we see below output.

Re: collect_corrupt_items_vacuum.patch

2022-11-07 Thread Nikita Malakhov
Hi hackers! Daniel is busy with other tasks. I've found this topic and this problem seems to be actual or v15 too. Please correct me if I am wrong. I've checked another discussion related to pg_visibility [1]. According to discussion: if using latest completed xid is not right for checking

Re: Postgres auto vacuum - Disable

2022-11-07 Thread Laurenz Albe
On Mon, 2022-11-07 at 12:12 +, Karthik Jagadish (kjagadis) wrote: > I have follow-up question where the vacuum process is waiting and not doing > it’s job. > When we grep on waiting process we see below output. Whenever we see this we > notice > that the vacuum is not happening and the

Re: refactor ownercheck and aclcheck functions

2022-11-07 Thread Antonin Houska
Peter Eisentraut wrote: > These patches take the dozens of mostly-duplicate pg_foo_ownercheck() and > pg_foo_aclcheck() functions and replace (most of) them by common functions > that are driven by the ObjectProperty table. All the required information is > already in that table. > > This is

Re: Postgres auto vacuum - Disable

2022-11-07 Thread Karthik Jagadish (kjagadis)
Hi, Thanks for the response. I have follow-up question where the vacuum process is waiting and not doing it’s job. When we grep on waiting process we see below output. Whenever we see this we notice that the vacuum is not happening and the system is running out of space. [root@zpah0031 ~]#

Re: Collation version tracking for macOS

2022-11-07 Thread Peter Eisentraut
On 02.11.22 00:57, Thomas Munro wrote: 3. Library availability. This is a problem for downstream communities to solve. For example, the people who build Windows installers might want to start bundling the ICU versions from their earlier releases, the people involved with each Linux/BSD distro

Re: pg_upgrade test failure

2022-11-07 Thread Thomas Munro
So [1] on its own didn't fix this. My next guess is that the attached might help. Hmm. Following Michael's clue that this might involve log files and pg_ctl, I noticed one thing: pg_ctl implements wait_for_postmaster_stop() by waiting for kill(pid, 0) to fail, and our kill emulation does

Re: Postgres auto vacuum - Disable

2022-11-07 Thread Dave Page
Hi On Mon, 7 Nov 2022 at 11:42, Karthik Jagadish (kjagadis) wrote: > Hi, > > > > We have a NMS application in cisco and using postgres as a database. > > > > We have query related to disabling auto vacuum. We have below > configuration in postgres.conf where the autovacuum=on is commented out.

RE: Perform streaming logical transactions by background workers and parallel apply

2022-11-07 Thread Hayato Kuroda (Fujitsu)
Dear Hou, The followings are my comments. I want to consider the patch more, but I sent it once. === worker.c 01. typedef enum TransApplyAction ``` /* * What action to take for the transaction. * * TRANS_LEADER_APPLY means that we are in the leader apply worker and changes * of the

Postgres auto vacuum - Disable

2022-11-07 Thread Karthik Jagadish (kjagadis)
Hi, We have a NMS application in cisco and using postgres as a database. We have query related to disabling auto vacuum. We have below configuration in postgres.conf where the autovacuum=on is commented out. [Shape Description automatically generated] But when checked in database we notice

Re: thinko in basic_archive.c

2022-11-07 Thread Bharath Rupireddy
On Sun, Nov 6, 2022 at 3:17 AM Nathan Bossart wrote: > > On Fri, Oct 21, 2022 at 09:30:16PM +0530, Bharath Rupireddy wrote: > > On Fri, Oct 21, 2022 at 10:43 AM Kyotaro Horiguchi > > wrote: > >> Thanks, but we don't need to wipe out the all bytes. Just putting \0 > >> at the beginning of the

Re: Error for WITH options on partitioned tables

2022-11-07 Thread Simon Riggs
On Mon, 7 Nov 2022 at 08:55, Karina Litskevich wrote: > > Hi David, > > > I am not very clear about why `build_reloptions` is removed in patch > > `v2-0002-better-error-message-for-setting-parameters-for-p.patch`, if > > you can help explain would be great. > > "build_reloptions" parses

Re: New docs chapter on Transaction Management and related changes

2022-11-07 Thread Simon Riggs
On Mon, 7 Nov 2022 at 07:43, Bruce Momjian wrote: > > On Fri, Nov 4, 2022 at 04:17:28PM +0100, Laurenz Albe wrote: > > On Sat, 2022-10-15 at 21:08 -0400, Bruce Momjian wrote: > > > I therefore merged all three paragraphs into > > > one and tried to make the text saner; release_savepoint.sgml

Re: Pluggable toaster

2022-11-07 Thread Aleksander Alekseev
Hi Nikita, > Pluggable TOAST is provided as an interface to allow developers to plug > in custom TOAST mechanics. It does not determines would it be universal > Toaster or one data type, but syntax for universal Toaster is out of scope > for this patchset. If I understand correctly, what is

Re: Perform streaming logical transactions by background workers and parallel apply

2022-11-07 Thread Masahiko Sawada
On Thu, Nov 3, 2022 at 10:06 PM houzj.f...@fujitsu.com wrote: > > On Wednesday, November 2, 2022 10:50 AM Masahiko Sawada > wrote: > > > > On Mon, Oct 24, 2022 at 8:42 PM Masahiko Sawada > > wrote: > > > > > > On Wed, Oct 12, 2022 at 3:04 PM Amit Kapila > > wrote: > > > > > > > > On Tue, Oct

Re: Adding doubly linked list type which stores the number of items in the list

2022-11-07 Thread Aleksander Alekseev
Hi Bharath, > > Regarding the improvement of the code coverage I realized that this > > may be a good patch for a newcomer. I know several people who may be > > interested in starting to contribute to PostgreSQL. Maybe I'll be able > > to find a volunteer. > > Hm. Or adding a ToDo item here

Re: Adding doubly linked list type which stores the number of items in the list

2022-11-07 Thread Bharath Rupireddy
On Mon, Nov 7, 2022 at 2:37 PM Aleksander Alekseev wrote: > > Regarding the improvement of the code coverage I realized that this > may be a good patch for a newcomer. I know several people who may be > interested in starting to contribute to PostgreSQL. Maybe I'll be able > to find a volunteer.

Use proc instead of MyProc in ProcArrayGroupClearXid()/TransactionGroupUpdateXidStatus()

2022-11-07 Thread rajesh singarapu
Hi, In both TransactionGroupUpdateXidStatus and ProcArrayGroupClearXid global MyProc is used. for consistency, replaced with a function local variable. thanks Rajesh v1-0001-Use-proc-instead-of-MyProc-in-ProcArrayGroupClear.patch Description: Binary data

Re: Adding doubly linked list type which stores the number of items in the list

2022-11-07 Thread Aleksander Alekseev
Hi Bharath, > Thanks for the patch. IMO, this can be discussed in a separate thread > to get more thoughts from the hackers. OK, I started a new thread [1], thanks. Regarding the improvement of the code coverage I realized that this may be a good patch for a newcomer. I know several people who

Re: [patch] Adding an assertion to report too long hash table name

2022-11-07 Thread Zhang Mingli
Hi, On Sep 29, 2022, 09:38 +0800, Xiaoran Wang , wrote: > Hi, > The max size for the shmem hash table name is SHMEM_INDEX_KEYSIZE - 1. but > when the caller uses a longer hash table name, it doesn't report any error, > insteadit just uses the first SHMEM_INDEX_KEYSIZE -1 chars as the hash

[PATCH] Const'ify the arguments of ilist.c/ilist.h functions

2022-11-07 Thread Aleksander Alekseev
Hi hackers, During the [1] discussion it was suggested to constify the arguments of ilist.c/ilist.h functions. Bharath (cc'ed) pointed out that it's better to start a new thread in order to attract more hackers that may be interested in this change, so I started one. The patch is attached. Here

  1   2   >