Re: PostgreSQL 17 Beta 1 release announcement draft

2024-05-19 Thread David Rowley
On Mon, 20 May 2024 at 09:35, Jonathan S. Katz wrote: > Thanks for all the feedback to date. Please see the next revision. > Again, please provide feedback no later than 2024-05-22 18:00 UTC. Thanks for the updates. > [`COPY`](https://www.postgresql.org/docs/17/sql-copy.html) is used to >

Re: First draft of PG 17 release notes

2024-05-18 Thread David Rowley
On Sun, 19 May 2024 at 02:40, Bruce Momjian wrote: > > On Thu, May 16, 2024 at 03:35:17PM +1200, David Rowley wrote: > > "Additionally, vacuum no longer silently imposes a 1GB tuple reference > > limit even when maintenance_work_mem or autovacuum_work_mem are set to > &g

Minor fixes for couple some comments around MERGE RETURNING

2024-05-18 Thread David Rowley
I noticed that PlannedStmt.hasReturning and hasModifyingCTE have an outdated comment now that MERGE supports RETURNING (per commit c649fa24a) i.e. these two: > bool hasReturning; /* is it insert|update|delete RETURNING? */ > bool hasModifyingCTE; /* has insert|update|delete in WITH? */

Re: generic plans and "initial" pruning

2024-05-18 Thread David Rowley
On Sun, 19 May 2024 at 13:27, Tom Lane wrote: > > David Rowley writes: > > 1. No ability to control the order that the locks are obtained. The > > order in which the locks are taken will be at the mercy of the plan > > the planner chooses. > > I do not thin

Re: generic plans and "initial" pruning

2024-05-18 Thread David Rowley
On Fri, 20 Jan 2023 at 08:39, Tom Lane wrote: > I spent some time re-reading this whole thread, and the more I read > the less happy I got. We are adding a lot of complexity and introducing > coding hazards that will surely bite somebody someday. And after awhile > I had what felt like an

Re: Incorrect Assert in BufFileSize()?

2024-05-17 Thread David Rowley
On Thu, 16 May 2024 at 07:20, Peter Geoghegan wrote: > > On Tue, May 14, 2024 at 6:58 AM David Rowley wrote: > > On Fri, 3 May 2024 at 16:03, David Rowley wrote: > > > I'm trying to figure out why BufFileSize() Asserts that file->fileset > > > isn't NULL, pe

Re: Docs: Always use true|false instead of sometimes on|off for the subscription options

2024-05-16 Thread David Rowley
On Thu, 16 May 2024 at 19:05, Peter Smith wrote: > > On Thu, May 16, 2024 at 3:11 PM David Rowley wrote: > > If you want to do this, what's the reason to limit it to just this one > > page of the docs? > > Yeah, I had a vested interest in this one place because I've be

Re: PostgreSQL 17 Beta 1 release announcement draft

2024-05-16 Thread David Rowley
On Thu, 16 May 2024 at 17:37, zaidagilist wrote: > I am trying to open the 17 docs but it looks removed. Getting > following message "Page not found" > > https://www.postgresql.org/docs/17/ It's called "devel" for "development" until we branch sometime before July:

Re: Docs: Always use true|false instead of sometimes on|off for the subscription options

2024-05-15 Thread David Rowley
On Thu, 16 May 2024 at 12:29, Peter Smith wrote: > There are lots of subscription options listed on the CREATE > SUBSCRIPTION page [1]. > > Although these boolean options are capable of accepting different > values like "1|0", "on|off", "true|false", here they are all described > only using

Re: First draft of PG 17 release notes

2024-05-15 Thread David Rowley
On Thu, 16 May 2024 at 14:48, Bruce Momjian wrote: > > On Wed, May 15, 2024 at 09:13:14AM -0400, Melanie Plageman wrote: > > Also +1 on the Sawada/Naylor change being on the highlight section of > > the release (as David suggested upthread). > > Agreed, I went with the attached applied patch.

Re: PostgreSQL 17 Beta 1 release announcement draft

2024-05-15 Thread David Rowley
Thanks for writing that up. It's always exciting to see this each year. On Thu, 16 May 2024 at 13:45, Jonathan S. Katz wrote: > * Please indicate if you believe there's a notable omission, or if we > should omit any of these callouts I'd say the streaming read stuff added in b5a9b18cd and

Re: explain format json, unit for serialize and memory are different.

2024-05-15 Thread David Rowley
On Wed, 15 May 2024 at 13:23, Tom Lane wrote: > > Michael Paquier writes: > > Perhaps Alvaro and Tom would like to chime in, as committers of > > respectively 5de890e3610d and 06286709ee06? > > No objection here. In a green field I might argue for > round-to-nearest instead of round-up, but it

Re: Fixup a few 2023 copyright years

2024-05-15 Thread David Rowley
On Wed, 15 May 2024 at 17:32, Michael Paquier wrote: > While running src/tools/copyright.pl, I have noticed that that a > newline was missing at the end of index_including.sql, as an effect of > the test added by you in a63224be49b8. I've cleaned up that while on > it, as it was getting added

Re: First draft of PG 17 release notes

2024-05-15 Thread David Rowley
On Wed, 15 May 2024 at 20:38, Alvaro Herrera wrote: > > On 2024-May-14, Bruce Momjian wrote: > > I don't think users really care about these details, just that it is > > faster so they will not be surprised if there is a change. I was > > purposely vague to group multiple commits into the single

Re: explain format json, unit for serialize and memory are different.

2024-05-14 Thread David Rowley
On Wed, 15 May 2024 at 15:40, jian he wrote: > I am looking for an example where this information under json key > "Serialization" is not zero. > So far I have tried: Something that requires detoasting. > create table s(a text); > insert into s select repeat('a', 1024) from

Re: Fixup a few 2023 copyright years

2024-05-14 Thread David Rowley
On Tue, 9 Apr 2024 at 15:26, Michael Paquier wrote: > I would suggest to also wait until we're clearer with the situation > for all these mechanical changes, which I suspect is going to take 1~2 > weeks at least. Since the Oid resequencing and pgindent run is now done, I've pushed this patch.

Re: First draft of PG 17 release notes

2024-05-14 Thread David Rowley
On Wed, 15 May 2024 at 14:06, Bruce Momjian wrote: > I can use your wording, but how much prefetching to we enable now? I believe the read stream API is used for Seq Scan, ANALYZE and pg_prewarm(). fadvise() is used when the next buffer that's required is not in shared buffers on any build that

Re: explain format json, unit for serialize and memory are different.

2024-05-14 Thread David Rowley
On Wed, 15 May 2024 at 13:44, jian he wrote: >"Shared Hit Blocks": 0, >"Shared Read Blocks": 0, >"Shared Dirtied Blocks": 0, >"Shared Written Blocks": 0, >"Local Hit Blocks": 0, >"Local Read Blocks": 0, >"Local Dirtied Blocks": 0, >

Re: First draft of PG 17 release notes

2024-05-14 Thread David Rowley
On Wed, 15 May 2024 at 13:00, Bruce Momjian wrote: > > On Tue, May 14, 2024 at 03:39:26PM -0400, Melanie Plageman wrote: > > "Reduce system calls by automatically merging reads up to io_combine_limit" > > Uh, as I understand it, the reduced number of system calls is not the > value of the

Re: explain format json, unit for serialize and memory are different.

2024-05-14 Thread David Rowley
On Wed, 15 May 2024 at 01:18, jian he wrote: > else > { > ExplainPropertyInteger("Memory Used", "bytes", >mem_counters->totalspace - mem_counters->freespace, >es); > ExplainPropertyInteger("Memory Allocated", "bytes", >mem_counters->totalspace, es); > } > } > > the "else" branch, also

Re: Incorrect Assert in BufFileSize()?

2024-05-14 Thread David Rowley
On Fri, 3 May 2024 at 16:03, David Rowley wrote: > I'm trying to figure out why BufFileSize() Asserts that file->fileset > isn't NULL, per 1a990b207. I was hoping to get some feedback here. Here's a patch to remove the Assert(). Changing it to Assert(file->files != NULL); doesn't

Re: explain format json, unit for serialize and memory are different.

2024-05-14 Thread David Rowley
On Tue, 14 May 2024 at 18:16, David Rowley wrote: > I think for v17, we should consider adding a macro to explain.c to > calculate the KB from bytes. There are other inconsistencies that it > would be good to address. We normally round up to the nearest kilobyte > with (bytes +

Re: JIT compilation per plan node

2024-05-14 Thread David Rowley
On Tue, 14 May 2024 at 19:56, Jelte Fennema-Nio wrote: > I'm not saying I'd prefer the extra walk, but I don't think you'd need > to do this extra walk for all plans. Afaict you could skip the extra > walk when top_plan->total_cost < jit_above_cost. i.e. only doing the > extra walk to determine

Re: explain format json, unit for serialize and memory are different.

2024-05-14 Thread David Rowley
On Tue, 14 May 2024 at 17:40, Michael Paquier wrote: > > On Mon, May 13, 2024 at 11:22:08AM +0200, Daniel Gustafsson wrote: > > Since json (and yaml/xml) is intended to be machine-readable I think we use > > a > > single unit for all values, and document this fact. > > Agreed with the

Re: JIT compilation per plan node

2024-05-13 Thread David Rowley
On Fri, 15 Mar 2024 at 10:13, Tomas Vondra wrote: > To clarify, I think the patch is a step in the right direction, and a > meaningful improvement. It may not be the perfect solution we imagine > (but who knows how far we are from that), but AFAIK moving these > decisions to the node level is

Re: I have an exporting need...

2024-05-13 Thread David Rowley
On Tue, 14 May 2024 at 06:18, Juan Hernández wrote: > Do you consider useful to add a parameter (for example, --separatetables) so > when used the exporting file process can create a different tablename.sql > file for each table in database automatically? > > Example... > > PGHOST="/tmp"

Re: Why is parula failing?

2024-05-13 Thread David Rowley
On Thu, 21 Mar 2024 at 13:53, David Rowley wrote: > > On Thu, 21 Mar 2024 at 12:36, Tom Lane wrote: > > So yeah, if we could have log_autovacuum_min_duration = 0 perhaps > > that would yield a clue. > > FWIW, I agree with your earlier statement about it looking very muc

Re: 039_end_of_wal: error in "xl_tot_len zero" test

2024-05-12 Thread David Rowley
On Mon, 6 May 2024 at 15:06, Tom Lane wrote: > My best guess is that that changed the amount of WAL generated by > initdb just enough to make the problem reproduce on this animal. > However, why's it *only* happening on this animal? The amount of > WAL we generate isn't all that system-specific.

Re: First draft of PG 17 release notes

2024-05-10 Thread David Rowley
cheap startup plans when querying partitioned tables, inheritance parents and for UNION ALL (Andy Fan, David Rowley) Both under "E.1.3.1.1. Optimizer" David

Re: Support tid range scan in parallel?

2024-05-10 Thread David Rowley
On Fri, 10 May 2024 at 05:16, Cary Huang wrote: > I understand that the regression tests for parallel ctid range scan is a > bit lacking now. It only has a few EXPLAIN clauses to ensure parallel > workers are used when tid ranges are specified. They are added as > part of select_parallel.sql

Re: Use generation memory context for tuplestore.c

2024-05-10 Thread David Rowley
Thanks for having a look at this. On Sat, 11 May 2024 at 04:34, Dmitry Dolgov <9erthali...@gmail.com> wrote: > Do I understand correctly, that the efficiency of generation memory > context could be measured directly via counting number of malloc/free > calls? In those experiments I've conducted

Re: First draft of PG 17 release notes

2024-05-08 Thread David Rowley
On Thu, 9 May 2024 at 16:47, Muhammad Ikram wrote: > A minor formatting issue in the start below. Bullet is not required here. This is a placeholder for the highlight features of v17 will go. Bruce tends not to decide what those are all by himself. David

Re: First draft of PG 17 release notes

2024-05-08 Thread David Rowley
On Thu, 9 May 2024 at 16:04, Bruce Momjian wrote: > I welcome feedback. For some reason it was an easier job than usual. Thanks for working on that. > +2023-11-02 [cac169d68] Increase DEFAULT_FDW_TUPLE_COST from 0.01 to 0.2 > +Double the default foreign data wrapper tuple cost (Dav

Re: Expand applicability of aggregate's sortop optimization

2024-05-08 Thread David Rowley
On Thu, 9 May 2024 at 13:08, David Rowley wrote: > One additional thought is that the above method would also help > eliminate redundant sorting in queries with a GROUP BY clause. > Whereas, the can_minmax_aggs optimisation is not applied in that case. Another argument for using th

Re: Support tid range scan in parallel?

2024-05-08 Thread David Rowley
On Thu, 9 May 2024 at 10:23, Cary Huang wrote: > The v3 patch is attached. I've not looked at the patch, but please add it to the July CF. I'll try and look in more detail then. David

Re: Expand applicability of aggregate's sortop optimization

2024-05-08 Thread David Rowley
On Thu, 9 May 2024 at 12:26, David Rowley wrote: > I wonder if we should also consider as an alternative to this to just > have an aggregate support function, similar to > SupportRequestOptimizeWindowClause that just nullifies the aggorder / > aggdistinct fields for Min/Max aggrega

Re: Expand applicability of aggregate's sortop optimization

2024-05-08 Thread David Rowley
On Wed, 8 May 2024 at 22:13, Matthias van de Meent wrote: > As you may know, aggregates like SELECT MIN(unique1) FROM tenk1; are > rewritten as SELECT unique1 FROM tenk1 ORDER BY unique1 USING < LIMIT > 1; by using the optional sortop field in the aggregator. > However, this optimization is

Re: Revert: Remove useless self-joins *and* -DREALLOCATE_BITMAPSETS make server crash, regress test fail.

2024-05-08 Thread David Rowley
On Thu, 9 May 2024 at 06:24, Tom Lane wrote: > I like alternative-2.patch a lot better, not least because it > only adds cycles when join removal actually fires. Basically > this is putting the onus on the data structure modifier to > cope with shared bitmapsets, rather than trying to say that >

Re: Revert: Remove useless self-joins *and* -DREALLOCATE_BITMAPSETS make server crash, regress test fail.

2024-05-08 Thread David Rowley
On Thu, 9 May 2024 at 06:49, Tom Lane wrote: > BTW, now that I've wrapped my head around what's happening here, > I believe that -DREALLOCATE_BITMAPSETS is introducing a bug where > there was none before. The changes that left-join removal makes > won't cause any of these sets to go to empty, so

Re: 2024-05-09 release announcement draft

2024-05-08 Thread David Rowley
On Thu, 9 May 2024 at 04:17, Jonathan S. Katz wrote: > * Fix how > [`INSERT`](https://www.postgresql.org/docs/current/sql-insert.html) > handles multiple > [`VALUES`](https://www.postgresql.org/docs/current/sql-values.html) rows > into a target column that is a domain over an array or composite

Re: Revert: Remove useless self-joins *and* -DREALLOCATE_BITMAPSETS make server crash, regress test fail.

2024-05-07 Thread David Rowley
On Wed, 8 May 2024 at 10:55, Tom Lane wrote: > > David Rowley writes: > > REALLOCATE_BITMAPSETS was invented for this and IMO, it found exactly > > the problem it was invented to find. > > Not in a way that gives me any confidence that we found *all* the > problems.

Re: Revert: Remove useless self-joins *and* -DREALLOCATE_BITMAPSETS make server crash, regress test fail.

2024-05-07 Thread David Rowley
On Wed, 8 May 2024 at 10:40, Tom Lane wrote: > > David Rowley writes: > > We could make the policy copy-on-modify. If you put bms_copy around > > the bms_del_member() calls in remove_rel_from_query(), does it pass > > then? > > Didn't test, but that route seems awf

Re: Revert: Remove useless self-joins *and* -DREALLOCATE_BITMAPSETS make server crash, regress test fail.

2024-05-07 Thread David Rowley
On Wed, 8 May 2024 at 10:35, David Rowley wrote: > > On Wed, 8 May 2024 at 06:20, Tom Lane wrote: > > I find that Richard's proposed fix makes the core regression tests > > pass, but we still fail check-world. So I'm afraid we need something > > more aggressive, like

Re: Revert: Remove useless self-joins *and* -DREALLOCATE_BITMAPSETS make server crash, regress test fail.

2024-05-07 Thread David Rowley
On Wed, 8 May 2024 at 06:20, Tom Lane wrote: > I find that Richard's proposed fix makes the core regression tests > pass, but we still fail check-world. So I'm afraid we need something > more aggressive, like the attached which makes make_restrictinfo > copy all its input bitmapsets. Without

Re: Revert: Remove useless self-joins *and* -DREALLOCATE_BITMAPSETS make server crash, regress test fail.

2024-05-06 Thread David Rowley
On Tue, 7 May 2024 at 17:28, Tom Lane wrote: > > David Rowley writes: > > Yeah, before the revert, that did: > > - sjinf->syn_lefthand = replace_relid(sjinf->syn_lefthand, relid, > > subst); > > That replace code seems to have always done a bms_copy(

Re: Revert: Remove useless self-joins *and* -DREALLOCATE_BITMAPSETS make server crash, regress test fail.

2024-05-06 Thread David Rowley
On Tue, 7 May 2024 at 15:35, jian he wrote: > i also found that not specifying c_args: `-DREALLOCATE_BITMAPSETS`, > the regress test won't fail. It would be good to get some build farm coverage of this so we don't have to rely on manual testing. I wonder if it's a good idea to just define

Re: Revert: Remove useless self-joins *and* -DREALLOCATE_BITMAPSETS make server crash, regress test fail.

2024-05-06 Thread David Rowley
On Tue, 7 May 2024 at 17:11, David Rowley wrote: > sjinf->syn_lefthand = bms_del_member(sjinf->syn_lefthand, relid); > > I've not looked, but I assumed the revert must have removed some > common code that was added and reverted with SJE. Yeah, before the revert, that d

Re: Revert: Remove useless self-joins *and* -DREALLOCATE_BITMAPSETS make server crash, regress test fail.

2024-05-06 Thread David Rowley
On Tue, 7 May 2024 at 17:01, Tom Lane wrote: > > Richard Guo writes: > > Thank you for the report. I looked at this a little bit and I think > > here is what happened. In deconstruct_distribute_oj_quals we call > > distribute_quals_to_rels using the uncopied sjinfo->syn_lefthand as > >

Re: Revert: Remove useless self-joins *and* -DREALLOCATE_BITMAPSETS make server crash, regress test fail.

2024-05-06 Thread David Rowley
On Tue, 7 May 2024 at 16:47, Richard Guo wrote: > --- a/src/backend/optimizer/plan/initsplan.c > +++ b/src/backend/optimizer/plan/initsplan.c > @@ -1888,7 +1888,7 @@ deconstruct_distribute_oj_quals(PlannerInfo *root, > qualscope = bms_union(sjinfo->syn_lefthand, sjinfo->syn_righthand); >

Re: 2024-05-09 release announcement draft

2024-05-06 Thread David Rowley
On Tue, 7 May 2024 at 15:48, Tom Lane wrote: > > David Rowley writes: > > I know this is the same wording as Tom added in [1], I might just have > > failed to comprehend something, but if I strip out the links and try > > to make sense of "Fix INSERT from multi

Re: 2024-05-09 release announcement draft

2024-05-06 Thread David Rowley
On Tue, 7 May 2024 at 15:09, Jonathan S. Katz wrote: > I opted for that; and it turned out the other fix was simple, so here's > an updated draft. Thanks > * Fix [`INSERT`](https://www.postgresql.org/docs/current/sql-insert.html) from > multiple

Re: 2024-05-09 release announcement draft

2024-05-06 Thread David Rowley
On Tue, 7 May 2024 at 14:58, Jonathan S. Katz wrote: > >> * Throw an error if an index is accessed while it is being reindexed. > > > > Based on this, I'd vote to just remove it from the release announcement. I'd prefer that over leaving the wording the way it is. Looking at the test case in

Re: Incorrect explain output for updates/delete operations with returning-list on partitioned tables

2024-05-06 Thread David Rowley
On Tue, 7 May 2024 at 09:18, Tom Lane wrote: > > SAIKIRAN AVULA writes: > > I have been working on partitioned tables recently, and I have noticed > > something that doesn't seem correct with the EXPLAIN output of an > > update/delete query with a returning list. > > What do you think is not

Re: 2024-05-09 release announcement draft

2024-05-06 Thread David Rowley
On Tue, 7 May 2024 at 05:44, Jonathan S. Katz wrote: > Please provide feedback no later (and preferably sooner) than 2024-05-09 > 12:00 UTC. Thanks for the draft. Here's some feedback. > * Fix [`INSERT`](https://www.postgresql.org/docs/current/sql-insert.html) from > multiple

Re: On disable_cost

2024-05-04 Thread David Rowley
On Sun, 5 May 2024 at 04:57, Tom Lane wrote: > > David Rowley writes: > > That doesn't get you the benefits of fewer CPU cycles, but where did > > that come from as a motive to change this? There's no shortage of > > other ways to make the planner faster if that's a

Re: On disable_cost

2024-05-04 Thread David Rowley
On Sat, 4 May 2024 at 08:34, Robert Haas wrote: > Another idea is to remove the ERROR mentioned above from > set_cheapest() and just allow planning to continue even if some > relations end up with no paths. (This would necessitate finding and > fixing any code that could be confused by a pathless

Re: Use generation memory context for tuplestore.c

2024-05-03 Thread David Rowley
On Sat, 4 May 2024 at 03:51, Matthias van de Meent wrote: > Was a bump context considered? If so, why didn't it make the cut? > If tuplestore_trim is the only reason why the type of context in patch > 2 is a generation context, then couldn't we make the type of context > conditional on

Re: Support tid range scan in parallel?

2024-05-03 Thread David Rowley
On Sat, 4 May 2024 at 06:55, Cary Huang wrote: > With syncscan enabled, the "table_block_parallelscan_nextpage()" would > return the next block since the end of the first tid rangescan instead of the > correct start block that should be scanned. I see that single tid rangescan > does not have

Re: Typos in the code and README

2024-05-03 Thread David Rowley
On Fri, 3 May 2024 at 00:00, Alexander Lakhin wrote: > (some of them are located in doc/, so it's not a code-only change) > I've attached the patch for your convenience, though maybe some > of the suggestions are to be discarded. Thanks. I was hoping you'd do that. I pushed the patch after only

Use generation memory context for tuplestore.c

2024-05-03 Thread David Rowley
(40af10b57 did this for tuplesort.c, this is the same, but for tuplestore.c) I was looking at the tuplestore.c code a few days ago and noticed that it allocates tuples in the memory context that tuplestore_begin_heap() is called in, which for nodeMaterial.c, is ExecutorState. I didn't think this

Incorrect Assert in BufFileSize()?

2024-05-02 Thread David Rowley
I'm trying to figure out why BufFileSize() Asserts that file->fileset isn't NULL, per 1a990b207. The discussion for that commit is in [1], but I don't see any explanation of the Assert in the discussion or commit message and there's no comment explaining why it's there. The code that comes after

Re: enhance the efficiency of migrating particularly large tables

2024-05-02 Thread David Rowley
On Fri, 3 May 2024 at 09:33, David Zhang wrote: > Is there a simple way to get the min of ctid faster than using min(), but > similar to get the max of ctid using pg_relation_size? The equivalent approximation is always '(0,1)'. David

Re: Type and CAST error on lowest negative integer values for smallint, int and bigint

2024-05-02 Thread David Rowley
On Thu, 2 May 2024 at 23:25, Hans Buschmann wrote: > postgres=# select -32768::smallint; > ERROR: smallint out of range The precedence order of operations applies the cast before the unary minus operator. Any of the following will work: postgres=# select cast(-32768 as smallint),

Re: Document NULL

2024-05-01 Thread David Rowley
On Thu, 2 May 2024 at 03:12, David G. Johnston wrote: > Attached is a very rough draft attempting this, based on my own thoughts and > those expressed by Tom in [1], which largely align with mine. Thanks for picking this up. I agree that we should have something to improve this. It would be

Re: New GUC autovacuum_max_threshold ?

2024-05-01 Thread David Rowley
On Sat, 27 Apr 2024 at 02:13, Robert Haas wrote: > Let's compare the current situation to the situation post-patch with a > cap of 500k. Consider a table 1024 times larger than the one I > mentioned above, so pgbench scale factor 25600, size on disk 320GB. > Currently, that table will be vacuumed

Re: Support tid range scan in parallel?

2024-05-01 Thread David Rowley
On Wed, 1 May 2024 at 07:10, Cary Huang wrote: > Yes of course. These numbers were obtained earlier this year on master with > the patch applied most likely without the read stream code you mentioned. The > patch attached here is rebased to commit > dd0183469bb779247c96e86c2272dca7ff4ec9e7 on

Re: Streaming I/O, vectored I/O (WIP)

2024-04-30 Thread David Rowley
On Wed, 24 Apr 2024 at 14:32, David Rowley wrote: > I've attached a patch with a few typo fixes and what looks like an > incorrect type for max_ios. It's an int16 and I think it needs to be > an int. Doing "max_ios = Min(max_ios, PG_INT16_MAX);" doesn't do > anything

Re: Support tid range scan in parallel?

2024-04-29 Thread David Rowley
On Tue, 30 Apr 2024 at 10:36, Cary Huang wrote: > In one of our migration scenarios, we rely on tid range scan to migrate huge > table from one database to another once the lower and upper ctid bound is > determined. With the support of parallel ctid range scan, this process could > be done

Re: Typos in the code and README

2024-04-28 Thread David Rowley
On Sat, 20 Apr 2024 at 16:09, David Rowley wrote: > Here are a few more to see if it motivates anyone else to do a more > thorough search for another batch. I've pushed these now. David

Re: Add memory context type to pg_backend_memory_contexts view

2024-04-23 Thread David Rowley
On Tue, 16 Apr 2024 at 13:30, David Rowley wrote: > In [1] Andres mentioned that there's no way to determine the memory > context type in pg_backend_memory_contexts. This is a bit annoying as > I'd like to add a test to exercise BumpStats(). > > Having the context type in the

Re: Streaming I/O, vectored I/O (WIP)

2024-04-23 Thread David Rowley
I've attached a patch with a few typo fixes and what looks like an incorrect type for max_ios. It's an int16 and I think it needs to be an int. Doing "max_ios = Min(max_ios, PG_INT16_MAX);" doesn't do anything when max_ios is int16. David diff --git a/src/backend/storage/aio/read_stream.c

Re: Minor document typo

2024-04-23 Thread David Rowley
On Wed, 24 Apr 2024 at 00:11, Tatsuo Ishii wrote: > Just out of a curiosity, is it possible to say "low a wal_level on the > primary"? (just "too" is removed) Prefixing the adjective with "too" means it's beyond the acceptable range. "This coffee is too hot".

Re: Minor document typo

2024-04-23 Thread David Rowley
On Tue, 23 Apr 2024 at 23:17, Tatsuo Ishii wrote: >Number of uses of logical slots in this database that have been >canceled due to old snapshots or too low a linkend="guc-wal-level"/> >on the primary > > I think "too low a" should be "too low" ('a' is not > necessary).

Re: Typos in the code and README

2024-04-19 Thread David Rowley
On Fri, 19 Apr 2024 at 20:13, Daniel Gustafsson wrote: > Thanks, I incorporated these into 0001 before pushing. All the commits in > this > patchset are now applied. Here are a few more to see if it motivates anyone else to do a more thorough search for another batch. Fixes duplicate words

Re: Stability of queryid in minor versions

2024-04-19 Thread David Rowley
On Tue, 16 Apr 2024 at 15:16, Michael Paquier wrote: > > On Tue, Apr 16, 2024 at 02:04:22PM +1200, David Rowley wrote: > > It makes sense to talk about the hashing variations closer to the > > object identifier part. > > Mostly what I had in mind. A separate for the n

Re: Why is parula failing?

2024-04-16 Thread David Rowley
On Tue, 16 Apr 2024 at 18:58, Robins Tharakan wrote: > The last 25 consecutive runs have passed [1] after switching > REL_12_STABLE to -O0 ! So I am wondering whether that confirms that > the compiler version is to blame, and while we're still here, > is there anything else I could try? I don't

Re: Add bump memory context type and use it for tuplesorts

2024-04-16 Thread David Rowley
On Mon, 8 Apr 2024 at 00:37, David Rowley wrote: > I've now pushed all 3 patches. Thank you for all the reviews on > these and for the extra MemoryContextMethodID bit, Matthias. I realised earlier today when working on [1] that bump makes a pretty brain-dead move when adding dedicated

Re: Add bump memory context type and use it for tuplesorts

2024-04-16 Thread David Rowley
On Tue, 16 Apr 2024 at 17:13, Amul Sul wrote: > Attached is a small patch adding the missing BumpContext description to the > README. Thanks for noticing and working on the patch. There were a few things that were not quite accurate or are misleading: 1. > +These three memory contexts aim to

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread David Rowley
On Tue, 16 Apr 2024 at 14:29, Andres Freund wrote: > I think total_nblocks might also not be entirely stable? I think it is stable for this test. However, I'll let the buildfarm make the final call on that. The reason I want to include it is that I'd like to push the large allocations to the

Re: Stability of queryid in minor versions

2024-04-15 Thread David Rowley
On Tue, 16 Apr 2024 at 12:10, Michael Paquier wrote: > Not sure that this is an improvement in clarity. There are a few > bullet points that treat about the instability of the query ID, and > your patch is now mixing the query ID being different for two > mostly-identical queries on the same

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread David Rowley
On Tue, 16 Apr 2024 at 10:57, Andres Freund wrote: > I guess was thinking more about BumpIsEmpty() and BumpStats() then the "bogus" > cases. But BumpIsEmpty() likely is unreachable as well. The only call to MemoryContextIsEmpty() I see is AtSubCommit_Memory() and it's on an aset.c context type.

Add memory context type to pg_backend_memory_contexts view

2024-04-15 Thread David Rowley
0a58697a4b88bc3ac80f09ed78b56ebe903a2aae Mon Sep 17 00:00:00 2001 From: David Rowley Date: Tue, 16 Apr 2024 13:05:34 +1200 Subject: [PATCH v1] Add context type field to pg_backend_memory_contexts --- doc/src/sgml/system-views.sgml | 9 + src/backend/utils/adt/mcxtfuncs.c | 50

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread David Rowley
On Mon, 15 Apr 2024 at 10:33, Andres Freund wrote: > - The new bump allocator has a fair amount of uncovered functionality: > > https://anarazel.de/postgres/cov/16-vs-HEAD-2024-04-14/src/backend/utils/mmgr/bump.c.gcov.html#L293 The attached adds a test to tuplesort to exercise

Re: Why is parula failing?

2024-04-14 Thread David Rowley
On Mon, 15 Apr 2024 at 16:10, Robins Tharakan wrote: > - I now have 2 separate runs stuck on pg_sleep() - HEAD / REL_16_STABLE > - I'll keep them (stuck) for this week, in case there's more we can get > from them (and to see how long they take) > - Attached are 'bt full' outputs for both (b.txt -

Re: Stability of queryid in minor versions

2024-04-14 Thread David Rowley
On Mon, 15 Apr 2024 at 14:09, Michael Paquier wrote: > > On Mon, Apr 15, 2024 at 01:31:47PM +1200, David Rowley wrote: > > I'm unsure if "Rule of thumb" is the correct way to convey that. We > > can't really write "We endeavour to", as who is "We&quo

Re: Fix out-of-bounds in the function GetCommandTagName

2024-04-14 Thread David Rowley
On Mon, 15 Apr 2024 at 12:12, Ranier Vilela wrote: > > Em dom., 14 de abr. de 2024 às 20:38, David Rowley > escreveu: >> You seem to have forgotten to attach it, but my comments above were >> written with the assumption that the patch is what I've attached here. >

Re: Stability of queryid in minor versions

2024-04-14 Thread David Rowley
On Mon, 15 Apr 2024 at 13:37, David G. Johnston wrote: > Seems we can improve things by simply removing the "rule of thumb" sentence > altogether. The prior paragraph states the things the queryid depends upon > at the level of detail the reader needs. I don't think that addresses the

Re: Stability of queryid in minor versions

2024-04-14 Thread David Rowley
On Mon, 15 Apr 2024 at 13:19, Tom Lane wrote: > > Michael Paquier writes: > > On Mon, Apr 15, 2024 at 11:20:16AM +1200, David Rowley wrote: > >> 1. We cannot change Node enums in minor versions > >> 2. We're *unlikely* to add fields to Node types in minor versions,

Re: Stability of queryid in minor versions

2024-04-14 Thread David Rowley
On Mon, 15 Apr 2024 at 12:04, Michael Paquier wrote: > Since 16 these new fields would be added by default unless the node > attribute query_jumble_ignore is appended to it. I agree that this > may not be entirely intuitive when it comes to force compatibility > across the same major version.

Re: Fix out-of-bounds in the function GetCommandTagName

2024-04-14 Thread David Rowley
On Mon, 15 Apr 2024 at 11:54, Tom Lane wrote: > would this also allow us to get rid of any default: > cases in switches on command tags? git grep "case CMDTAG_" does not yield any results. As far as I understand, we'd only be able to get rid of a default case if we had a switch that included

Re: Stability of queryid in minor versions

2024-04-14 Thread David Rowley
On Mon, 15 Apr 2024 at 11:47, Peter Geoghegan wrote: > > On Sun, Apr 14, 2024 at 7:20 PM David Rowley wrote: > > It's the "underlying server version" that I think needs some > > clarification. It's unclear if the minor version must match or just > > the m

Re: Fix out-of-bounds in the function GetCommandTagName

2024-04-14 Thread David Rowley
On Mon, 15 Apr 2024 at 11:17, Ranier Vilela wrote: > Coverity has reported some out-of-bounds bugs > related to the GetCommandTagName function. > > The size of the array is defined by COMMAND_TAG_NEXTTAG enum, > whose value currently corresponds to 193. > Since enum items are evaluated starting

Stability of queryid in minor versions

2024-04-14 Thread David Rowley
I was recently asked internally about the stability guarantees we offer for queryid. My answer consisted of: 1. We cannot change Node enums in minor versions 2. We're *unlikely* to add fields to Node types in minor versions, and if we did we'd likely be leaving them out of the jumble calc, plus

Re: Typos in the code and README

2024-04-14 Thread David Rowley
On Sat, 13 Apr 2024 at 09:17, Daniel Gustafsson wrote: > > > On 12 Apr 2024, at 23:15, Heikki Linnakangas wrote: > > Here's a few more. I've accumulate these over the past couple of months, > > keeping them stashed in a branch, adding to it whenever I've spotted a > > minor typo while reading

SupportRequestRows support function for generate_series_timestamptz

2024-04-13 Thread David Rowley
this doesn't help the case in [1] as the generate_series inputs are not const there) David [1] https://www.postgresql.org/message-id/CAMPYKo0FouB-HZ1k-_Ur2v%2BkK71q0T5icQGrp%2BSPbQJGq0H2Rw%40mail.gmail.com From ca0e982215d1335d015a2b515f038b7186935af0 Mon Sep 17 00:00:00 2001 From: David Rowley

Re: Incorrect handling of IS [NOT] NULL quals on inheritance parents

2024-04-12 Thread David Rowley
On Thu, 11 Apr 2024 at 18:49, Richard Guo wrote: > I agree with both of your points. But I also think we do not need to > skip applying the NOT NULL qual optimization for partitioned tables. > For partitioned tables, if the parent is marked NOT NULL, then all its > children must also be marked

Re: Typos in the code and README

2024-04-11 Thread David Rowley
On Fri, 12 Apr 2024, 1:05 am Daniel Gustafsson, wrote: > Now that the tree has settled down a bit post-freeze I ran some tooling to > check spelling. I was primarily interested in docs and README* which were > mostly free from simply typos, while the code had some in various comments > and >

Re: Incorrect handling of IS [NOT] NULL quals on inheritance parents

2024-04-10 Thread David Rowley
On Wed, 10 Apr 2024 at 19:12, Richard Guo wrote: > And I think recording NOT NULL columns for traditional inheritance > parents can be error-prone for some future optimization where we look > at an inheritance parent's notnullattnums and make decisions based on > the assumption that the included

Re: Incorrect handling of IS [NOT] NULL quals on inheritance parents

2024-04-09 Thread David Rowley
On Wed, 10 Apr 2024 at 17:18, Tom Lane wrote: > > David Rowley writes: > > I think a better fix is just to not apply the optimisation for > > inheritance RTEs in add_base_clause_to_rel(). > > Is it worth paying attention to whether the constraint is marked > conn

Re: Incorrect handling of IS [NOT] NULL quals on inheritance parents

2024-04-09 Thread David Rowley
On Tue, 9 Apr 2024 at 21:55, Richard Guo wrote: > > In b262ad440e we introduced an optimization that drops IS NOT NULL quals > on a NOT NULL column, and reduces IS NULL quals on a NOT NULL column to > constant-FALSE. I happened to notice that this is not working correctly > for traditional

  1   2   3   4   5   6   7   8   9   10   >