Re: New docs chapter on Transaction Management and related changes

2022-11-29 Thread Simon Riggs
On Wed, 30 Nov 2022 at 01:51, Bruce Momjian wrote: > Thanks to Simon for getting this important > information in our docs, and for the valuable feedback from others that > made this even better. And thanks to you for pulling that all together Bruce. -- Simon Riggs

Re: Slow standby snapshot

2022-11-29 Thread Simon Riggs
and pushed. That is a complete patch from multiple angles; very happy here. Thanks for a great job. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Reducing power consumption on idle servers

2022-11-29 Thread Simon Riggs
On Wed, 30 Nov 2022 at 03:50, Thomas Munro wrote: > > On Wed, Nov 30, 2022 at 1:32 AM Simon Riggs > wrote: > > Re-attaching patch for bgwriter and walwriter, so it is clear this is > > not yet committed. > > I'm just curious, and not suggesting that 60s wakeups a

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-29 Thread Simon Riggs
On Wed, 30 Nov 2022 at 03:56, Nathan Bossart wrote: > > On Tue, Nov 29, 2022 at 12:02:44PM +, Simon Riggs wrote: > > The last important point for me is tests, in src/test/modules > > probably. It might be possible to reuse the final state of other > > modules'

Re: Bug in wait time when waiting on nested subtransaction

2022-11-29 Thread Simon Riggs
avior, if we do. The really big issue is the loss of performance we get from having subtrans point only to its immediate parent, which makes XidInMVCCSnapshot() go really slow in the presence of lots of subtransactions. So ignoring the issue on this thread will open the door for the optimizati

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-29 Thread Simon Riggs
On Thu, 17 Nov 2022 at 17:29, Simon Riggs wrote: > (yes, the last line shows x10 performance patched, that is not a typo) New version of patch, now just a one-line patch! Results show it's still a good win for XidInMVCCSnapshot(). -- Simon Riggshttp://www.Enterpri

Re: Reducing power consumption on idle servers

2022-11-29 Thread Simon Riggs
On Sun, 20 Nov 2022 at 20:00, Simon Riggs wrote: > > On Thu, 24 Mar 2022 at 16:21, Robert Haas wrote: > > > > On Thu, Mar 24, 2022 at 12:02 PM Simon Riggs > > > > What changes will be acceptable for bgwriter, walwriter and logical > > > worker? > > &

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-29 Thread Simon Riggs
reuse the final state of other modules' tests to test cleanup, or at least integrate a custodian test into each module. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Reducing power consumption on idle servers

2022-11-29 Thread Simon Riggs
nuously busy duty cycle and most of them not in the cloud. If we guess that we save 10W when idle, then that saves some proportion of a GW. It's not a huge contribution to the effort, but if by doing this we inspire others to do the same, we may yet make a difference. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Bug in wait time when waiting on nested subtransaction

2022-11-28 Thread Simon Riggs
On Mon, 28 Nov 2022 at 17:38, Robert Haas wrote: > > On Mon, Nov 28, 2022 at 10:28 AM Simon Riggs > wrote: > > So we have these options > > > > 1. Removing the XactLockTableDelete() call in CommitSubTransaction(). > > That releases lock waiters earlier than exp

Re: Bug in wait time when waiting on nested subtransaction

2022-11-28 Thread Simon Riggs
On Mon, 28 Nov 2022 at 18:53, Alvaro Herrera wrote: > > On 2022-Nov-28, Simon Riggs wrote: > > > A narrative description of the issue follows: > > session1 - requests multiple nested subtransactions like this: > > BEGIN; ... > > SAVEPOINT subxid1; ... > > S

Bug in wait time when waiting on nested subtransaction

2022-11-28 Thread Simon Riggs
page(s) and ensure that a page is less often paged out when we are still using it. Thoughts? -- Simon Riggshttp://www.EnterpriseDB.com/ 001-isotest-tuplelock-subxact.v1.patch Description: Binary data

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-28 Thread Simon Riggs
On Sun, 27 Nov 2022 at 23:34, Nathan Bossart wrote: > > Thanks for taking a look! > > On Thu, Nov 24, 2022 at 05:31:02PM +, Simon Riggs wrote: > > * not sure I believe that everything it does can always be aborted out > > of and shutdown - to achi

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-24 Thread Simon Riggs
this gives us a way to have both stability and innovation at the same time? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-24 Thread Simon Riggs
e change in a later phase of Beta I can't really comment with knowledge on sub-patches 0002 to 0004. Perhaps you should aim to get 1, 5, 6 committed first and then return to the others in a later CF/separate thread? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-24 Thread Simon Riggs
loop does respond promptly > to control-C even in HEAD. So there are CFI(s) in the loop as > I thought, and we don't need another. Thanks for checking. Sorry for not responding earlier. > What we do need is some more work on nearby comments. I'll > see about that and

Re: Hash index build performance tweak from sorting

2022-11-23 Thread Simon Riggs
nts. > and v4 includes 7 extra lines in hashinsert.c for the Assert() I > mentioned in my previous email plus a bunch of extra comments. Oh, I did already include that in v3 as requested. > I'd rather see this solved like v4 is doing it. Please do. No further comments. Thanks fo

Re: Allow single table VACUUM in transaction block

2022-11-22 Thread Simon Riggs
On Tue, 22 Nov 2022 at 16:43, Justin Pryzby wrote: > > On Mon, Nov 21, 2022 at 03:07:25PM +, Simon Riggs wrote: > > Attached patch implements VACUUM (BACKGROUND). > > > > There are quite a few small details to consider; please read the docs > > and comments.

Re: Slow standby snapshot

2022-11-22 Thread Simon Riggs
On Tue, 22 Nov 2022 at 16:53, Tom Lane wrote: > > Simon Riggs writes: > > On Tue, 22 Nov 2022 at 16:28, Tom Lane wrote: > >> If we do those things, do we need a wasted-work counter at all? > > > The wasted work counter works well to respond to heavy read-only >

Re: Slow standby snapshot

2022-11-22 Thread Simon Riggs
On Tue, 22 Nov 2022 at 16:28, Tom Lane wrote: > > Simon Riggs writes: > > We seem to have replaced one magic constant with another, so not sure > > if this is autotuning, but I like it much better than what we had > > before (i.e. better than my prev patch). > > Yea

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-22 Thread Simon Riggs
ion didn't have the issue, but the latest seems to only > > limit after a visible tuple has been found. Note the continue; when > > fetching a heap tuple fails. > > Oh, that line was removed in Simon's patch but not in Jakub's version, > I guess. Jakub's versio

Re: Code checks for App Devs, using new options for transaction behavior

2022-11-22 Thread Simon Riggs
On Mon, 7 Nov 2022 at 14:25, Simon Riggs wrote: > > On Wed, 2 Nov 2022 at 07:40, Simon Riggs wrote: > > > > What will be the behavior if someone declares a savepoint with this > > > name ("_internal_nested_xact"). Will this interfere with this new > >

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-22 Thread Simon Riggs
ong series of deletions from the leftmost index pages. Not sure how, just framing the problem. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-22 Thread Simon Riggs
WIP to see how it might work. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Simon Riggs
een found. Note the continue; when fetching a heap > tuple fails. Agreed, resolved in this version. Robert, something like this perhaps? limit on both the index and the heap. -- Simon Riggshttp://www.EnterpriseDB.com/ 0001-Damage-control-for-planner-s-get_actual_variable_end.v2.patch Description: Binary data

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Simon Riggs
On Mon, 21 Nov 2022 at 15:23, Robert Haas wrote: > > On Mon, Nov 21, 2022 at 10:14 AM Simon Riggs > wrote: > > > > What we need is a solution that avoids reading an unbounded number of > > > > tuples under any circumstances. I previously suggested using > &g

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Simon Riggs
ll such a thing, should we undo any of the previous > changes that backed off the reliability of the result? Not sure. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Allow single table VACUUM in transaction block

2022-11-21 Thread Simon Riggs
On Fri, 18 Nov 2022 at 11:54, Simon Riggs wrote: > > On Thu, 17 Nov 2022 at 20:00, Justin Pryzby wrote: > > > > On Wed, Nov 16, 2022 at 05:14:07PM -0500, Greg Stark wrote: > > > I think this requesting autovacuum worker should be a distinct > > > command. Or a

Re: Allow single table VACUUM in transaction block

2022-11-21 Thread Simon Riggs
On Fri, 18 Nov 2022 at 18:26, Tom Lane wrote: > > Robert Haas writes: > > On Fri, Nov 18, 2022 at 7:04 AM Simon Riggs > > wrote: > >> So if consistency is also a strong requirement, then maybe we should > >> make that new command the default, i.e. make

Re: Patch: Global Unique Index

2022-11-21 Thread Simon Riggs
ay not actually be desirable in practice, or in a restricted use case. That means performance analysis of create, load, attach, detach, INSERT, SELECT, UPD/DEL and anything else that might be affected, together with algorithmic analysis of what happens for larger N and larger tables. Expect many versions; take provisions for many days. Best of luck -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Reducing power consumption on idle servers

2022-11-21 Thread Simon Riggs
On Mon, 21 Nov 2022 at 08:40, Laurenz Albe wrote: > > On Mon, 2022-11-21 at 07:36 +, Simon Riggs wrote: > > On Mon, 21 Nov 2022 at 05:07, Laurenz Albe wrote: > > > > > > On Mon, 2022-11-21 at 10:13 +1300, Thomas Munro wrote: > > > > I'll wait 2

Re: Reducing power consumption on idle servers

2022-11-20 Thread Simon Riggs
On Sun, 20 Nov 2022 at 22:55, Nathan Bossart wrote: > > On Mon, Nov 21, 2022 at 10:31:15AM +1300, Thomas Munro wrote: > > On Mon, Nov 21, 2022 at 9:00 AM Simon Riggs > > wrote: > >> As a 3rd patch, I will work on making logical workers hibernate. > > > > D

Re: Reducing power consumption on idle servers

2022-11-20 Thread Simon Riggs
promote, just enforcing a change to a better mechanism, hence I don't see a reason for a long(er) deprecation period than we have already had. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Reducing power consumption on idle servers

2022-11-20 Thread Simon Riggs
On Thu, 24 Mar 2022 at 16:21, Robert Haas wrote: > > On Thu, Mar 24, 2022 at 12:02 PM Simon Riggs > > What changes will be acceptable for bgwriter, walwriter and logical worker? > > Hmm, I think it would be fine to introduce some kind of hibernation > mechanism for logical

Re: Slow standby snapshot

2022-11-20 Thread Simon Riggs
ould also make sense to force compression when the startup process goes idle rather than wait for the work to be wasted first. Quick patch to add those two compression events also. That should favour the smaller wasted work limits. -- Simon Riggshttp://www.Enterpr

Re: Reducing power consumption on idle servers

2022-11-20 Thread Simon Riggs
On Sat, 19 Nov 2022 at 10:59, Simon Riggs wrote: > New version attached. Fix for doc xref -- Simon Riggshttp://www.EnterpriseDB.com/ hibernate_startup.v10.patch Description: Binary data

Re: Reducing power consumption on idle servers

2022-11-19 Thread Simon Riggs
On Fri, 18 Nov 2022 at 20:26, Thomas Munro wrote: > > On Sat, Nov 19, 2022 at 7:54 AM Simon Riggs > wrote: > > I agree. I can't see a reason to keep it anymore. > > +Use of promote_trigger_file is deprecated. If you're > > I think 'deprecated&#x

Re: Reducing power consumption on idle servers

2022-11-18 Thread Simon Riggs
On Thu, 17 Nov 2022 at 20:38, Robert Haas wrote: > > On Thu, Nov 17, 2022 at 2:55 AM Simon Riggs > wrote: > > No, it will have a direct effect only on people using promote_trigger_file > > who do not read and act upon the deprecation notice before upgrading > > by

Re: Allow single table VACUUM in transaction block

2022-11-18 Thread Simon Riggs
will be consistent. Can we at least have a vacuum_runs_in_background = on | off, to allow users to take advantage of this WITHOUT needing to rewrite all of their scripts? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Allow single table VACUUM in transaction block

2022-11-18 Thread Simon Riggs
ese: VACUUM (BACKGROUND) or VACUUM (AUTOVACUUM) - which might be clearer or VACUUM (ASYNC) - which is more descriptive of the behavior or we could go for both VACUUM (BACKGROUND, ASYNC) - since this allows us to have a BACKGROUND, SYNC version in the future Thoughts? -- Simon R

Re: Reducing power consumption on idle servers

2022-11-18 Thread Simon Riggs
On Fri, 18 Nov 2022 at 08:55, Bharath Rupireddy wrote: > However, I'm a bit > worried about how it'll affect the tools/providers/extensions that > depend on it. Who is that? Which ones depend upon it? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-18 Thread Simon Riggs
On Thu, 17 Nov 2022 at 20:29, Tomas Vondra wrote: > > On 11/17/22 18:29, Simon Riggs wrote: > > On Thu, 17 Nov 2022 at 17:04, Simon Riggs > > wrote: > >> > > 003 includes the idea to not-always do SubTransSetParent() > > > I'm a bit confused by th

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-17 Thread Simon Riggs
On Wed, 16 Nov 2022 at 15:44, Tom Lane wrote: > > Simon Riggs writes: > > On Wed, 16 Nov 2022 at 00:09, Tom Lane wrote: > >> No, that's not what XidInMVCCSnapshot does. If snapshot->suboverflowed > >> is set (ie, somebody somewh

Re: Reducing power consumption on idle servers

2022-11-16 Thread Simon Riggs
tty much everyone doing HA uses external HA software (cloud or otherwise) this shouldn't affect anyone. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Reducing power consumption on idle servers

2022-11-16 Thread Simon Riggs
On Wed, 16 Nov 2022 at 12:47, Bharath Rupireddy wrote: > > On Wed, Nov 16, 2022 at 2:34 PM Simon Riggs > wrote: > > > > Reposting v6 now so that patch tester doesn't think this has failed > > when the patch on other thread gets applied. > > Intentio

Re: Reducing power consumption on idle servers

2022-11-16 Thread Simon Riggs
On Sun, 13 Nov 2022 at 23:07, Simon Riggs wrote: > > On Sun, 13 Nov 2022 at 21:28, Thomas Munro wrote: > > > > On Mon, Nov 14, 2022 at 5:52 AM Simon Riggs > > wrote: > > > The attached patch is a reduced version of the original. It covers only: > > >

Re: when the startup process doesn't (logging startup delays)

2022-11-16 Thread Simon Riggs
CF > entry here - https://commitfest.postgresql.org/41/4012/. begin_startup_progress_phase() checks to see if feature is disabled twice, so I think you can skip the check and just rely on the check in enable(). Otherwise, all good. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Hash index build performance tweak from sorting

2022-11-15 Thread Simon Riggs
understand we often do this in > the code, but I feel like we do it less often in newer code. e.g we do > it in aset.c but not generation.c (which is much newer than aset.c). > My personal preference would be just to name the struct > HashInsertState and have no extra pointer typedefs. Not done, but not disagreeing either, just not very comfortable actually making those changes. -- Simon Riggshttp://www.EnterpriseDB.com/ hash_inserted_sorted.v3.patch Description: Binary data

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-15 Thread Simon Riggs
On Wed, 16 Nov 2022 at 00:09, Tom Lane wrote: > > Simon Riggs writes: > > On Tue, 15 Nov 2022 at 21:03, Tom Lane wrote: > >> The subxidStatus.overflowed check quoted above has a similar sort > >> of myopia: it's checking whether our current transaction has

Re: Slow standby snapshot

2022-11-15 Thread Simon Riggs
On Wed, 16 Nov 2022 at 00:15, Tom Lane wrote: > > Andres Freund writes: > > On 2022-11-15 23:14:42 +, Simon Riggs wrote: > >> Hence more frequent compression is effective at reducing the overhead. > >> But too frequent compression slows down the startup process,

Re: Slow standby snapshot

2022-11-15 Thread Simon Riggs
he overhead. But too frequent compression slows down the startup process, which can't then keep up. So we're just looking for an optimal frequency of compression for any given workload. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-15 Thread Simon Riggs
On Tue, 15 Nov 2022 at 21:03, Tom Lane wrote: > > Simon Riggs writes: > > New version attached. > > I looked at this patch and I do not see how it can possibly be safe. I grant you it is complex, so please bear with me. > The most direct counterexample aris

Re: when the startup process doesn't (logging startup delays)

2022-11-15 Thread Simon Riggs
On Tue, 15 Nov 2022 at 13:33, Bharath Rupireddy wrote: > > On Mon, Nov 14, 2022 at 9:31 PM Robert Haas wrote: > > > > On Mon, Nov 14, 2022 at 7:37 AM Simon Riggs > > wrote: > > > > Whilte at it, I noticed that we report redo progress for PITR, but we >

Re: New docs chapter on Transaction Management and related changes

2022-11-15 Thread Simon Riggs
On Tue, 8 Nov 2022 at 03:41, Bruce Momjian wrote: > > On Mon, Nov 7, 2022 at 10:58:05AM +, Simon Riggs wrote: > > What I've posted is the merged patch, i.e. your latest patch, plus > > changes to RELEASE SAVEPOINT from you on Oct 16, plus changes based on > > t

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-15 Thread Simon Riggs
On Mon, 7 Nov 2022 at 21:14, Simon Riggs wrote: > These results are compelling, thank you. > > Setting this to Ready for Committer. New version attached. -- Simon Riggshttp://www.EnterpriseDB.com/ 002_minimize_calls_to_SubTransSetParent.v12.patch Description: Binary data

Re: Allow single table VACUUM in transaction block

2022-11-15 Thread Simon Riggs
On Mon, 14 Nov 2022 at 19:52, Simon Riggs wrote: > > On Tue, 8 Nov 2022 at 03:10, Simon Riggs wrote: > > > > On Mon, 7 Nov 2022 at 08:20, Simon Riggs > > wrote: > > > > > Temp tables are actually easier, since we don't need any of the > &

Re: Allow single table VACUUM in transaction block

2022-11-14 Thread Simon Riggs
On Tue, 8 Nov 2022 at 03:10, Simon Riggs wrote: > > On Mon, 7 Nov 2022 at 08:20, Simon Riggs wrote: > > > Temp tables are actually easier, since we don't need any of the > > concurrency features we get with lazy vacuum. > Thoughts? New patch, which does this, when

Re: when the startup process doesn't (logging startup delays)

2022-11-14 Thread Simon Riggs
as to report on things that take long periods during recovery, not just prior to recovery. I would advocate that we disable progress only while waiting, as I've done here: https://www.postgresql.org/message-id/CANbhV-GcWjZ2cmj0uCbZDWQUHnneMi_4EfY3dVWq0-yD5o7Ccg%40mail.gmail.com -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Reducing power consumption on idle servers

2022-11-13 Thread Simon Riggs
On Sun, 13 Nov 2022 at 21:28, Thomas Munro wrote: > > On Mon, Nov 14, 2022 at 5:52 AM Simon Riggs > wrote: > > The attached patch is a reduced version of the original. It covers only: > > * deprecation of the promote_trigger_file - there are no tests that > > use that,

Re: Reducing power consumption on idle servers

2022-11-13 Thread Simon Riggs
On Thu, 24 Mar 2022 at 16:02, Simon Riggs wrote: > > On Thu, 24 Mar 2022 at 15:39, Robert Haas wrote: > > > > On Thu, Mar 24, 2022 at 6:59 AM Simon Riggs > > wrote: > > > The proposals of this patch are the following, each of which can be > > > indepen

Re: New docs chapter on Transaction Management and related changes

2022-11-10 Thread Simon Riggs
On Mon, 7 Nov 2022 at 22:04, Laurenz Albe wrote: > > On Sat, 2022-11-05 at 10:08 +, Simon Riggs wrote: > > Agreed; new compilation patch attached, including mine and then > > Robert's suggested rewordings. > > Thanks. There is clearly a lot of usefule informati

Re: Allow single table VACUUM in transaction block

2022-11-07 Thread Simon Riggs
On Mon, 7 Nov 2022 at 08:20, Simon Riggs wrote: > Temp tables are actually easier, since we don't need any of the > concurrency features we get with lazy vacuum. So the answer is to > always run a VACUUM FULL on temp tables since this skips any issues > with indexes etc.. So I

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-07 Thread Simon Riggs
t; > 120 177827 443202 > > > 128 399431 441842 > > > > > > From the performance point of view, this patch clearly fixes the > > > dramatic TPS collapse shown in these tests. > > > > I think these are really promising results. Although the perf result > > shows that the bottleneck on the SLRU is no more there with the patch, > > I think it would be nice to see the wait event as well. > > Please find attached samples returned by the following query when > testing transaction 1 with 80 subxacts: > SELECT wait_event_type, wait_event, locktype, mode, database, > relation, COUNT(*) from pg_stat_activity AS psa JOIN pg_locks AS pl ON > (psa.pid = pl.pid) GROUP BY 1, 2, 3, 4, 5, 6 ORDER BY 7 DESC; These results are compelling, thank you. Setting this to Ready for Committer. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Code checks for App Devs, using new options for transaction behavior

2022-11-07 Thread Simon Riggs
On Wed, 2 Nov 2022 at 07:40, Simon Riggs wrote: > > What will be the behavior if someone declares a savepoint with this > > name ("_internal_nested_xact"). Will this interfere with this new > > functionality? > > Clearly! I guess you are saying we should d

Re: Error for WITH options on partitioned tables

2022-11-07 Thread Simon Riggs
nores options, which are not in the list, while parsing. If > "validate" is true, it "elog"s ERROR when it meets option, which is not in the > allowed list. Karina's changes make sense to me, so +1. This is a minor patch, so I will set this as Ready For Committer. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: New docs chapter on Transaction Management and related changes

2022-11-07 Thread Simon Riggs
- > a later email. > > but that was unclear. Let me post one now, and Simon posted one too. What I've posted is the merged patch, i.e. your latest patch, plus changes to RELEASE SAVEPOINT from you on Oct 16, plus changes based on the later comments from Robert and I. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Allow single table VACUUM in transaction block

2022-11-07 Thread Simon Riggs
> work due to the presence of a long running snapshot. The VACUUM > doesn't have to dutifully report "success" when there is no practical > sense in which it was successful. There could be a back and forth > conversation between autovacuum.c and vacuumlazy.c that makes sure > that something useful happens sooner or later. The passage of time > really matters here. Regrettably, neither vacuum nor autovacuum waits for xmin to change; perhaps it should. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Allow single table VACUUM in transaction block

2022-11-06 Thread Simon Riggs
On Sun, 6 Nov 2022 at 18:50, Peter Geoghegan wrote: > > On Thu, Oct 27, 2022 at 2:31 AM Simon Riggs > wrote: > > Fix, so that this works without issue: > > > > BEGIN; > > > > VACUUM (ANALYZE) vactst; > > > > COMMIT; > > > >

Re: New docs chapter on Transaction Management and related changes

2022-11-05 Thread Simon Riggs
uding mine and then Robert's suggested rewordings. -- Simon Riggshttp://www.EnterpriseDB.com/ xact_docs.v9.patch Description: Binary data

Re: Allow single table VACUUM in transaction block

2022-11-04 Thread Simon Riggs
transaction block the vacuum scan can > +hold back the xmin horizon and does not update the database datfrozenxid, > +as a result this usage is not useful for database maintenance, but is > provided > +to allow vacuuming in special circumstances, such as temp

Re: Allow single table VACUUM in transaction block

2022-11-03 Thread Simon Riggs
On Tue, 1 Nov 2022 at 23:56, Simon Riggs wrote: > > I haven't checked the rest of the patch, but +1 for allowing VACUUM FULL > > within a user txn. > > My intention was to prevent that. I am certainly quite uneasy about > changing anything related to CLUSTER/VF, since t

Re: Code checks for App Devs, using new options for transaction behavior

2022-11-02 Thread Simon Riggs
On Wed, 2 Nov 2022 at 03:52, Dilip Kumar wrote: > > On Mon, Oct 31, 2022 at 6:54 PM Simon Riggs > wrote: > > > > > > What is the behavior if "nested_transactions" value is changed within > > > > a transaction execution, suppose the value was on a

Re: Error for WITH options on partitioned tables

2022-11-01 Thread Simon Riggs
uot; an ERROR with > > ERRCODE_WRONG_OBJECT_TYPE > > is reported). Then either the desired code should be passed to > > partitioned_table_reloptions() or similar checks and ereport calls > > should be > > placed in two different places. I'm not sure whether it is a good idea to > > change the code in one of these ways just to change the error code. > > > > Best regards, > > Karina Litskevich > > Postgres Professional: http://postgrespro.com/ > -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Allow single table VACUUM in transaction block

2022-11-01 Thread Simon Riggs
On Thu, 27 Oct 2022 at 21:07, Justin Pryzby wrote: > > On Thu, Oct 27, 2022 at 10:31:31AM +0100, Simon Riggs wrote: > > Allows both ANALYZE and vacuum of toast tables, but not VACUUM FULL. > > Maybe I misunderstood what you meant: you said "not VACUUM FULL", but &

Re: psql: Add command to use extended query protocol

2022-11-01 Thread Simon Riggs
On Tue, 1 Nov 2022 at 20:48, Peter Eisentraut wrote: > > On 01.11.22 10:10, Simon Riggs wrote: > > On Fri, 28 Oct 2022 at 07:53, Peter Eisentraut > > wrote: > >> > >> This adds a new psql command \gp that works like \g (or semicolon) but > >> uses

Re: psql: Add command to use extended query protocol

2022-11-01 Thread Simon Riggs
eral psql scripting. ...since if we used this in a script, it would be used like this, I think... SELECT $1, $2 \gp 'foo' 'bar' \gp 'bar' 'baz' ... -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Commit fest 2022-11

2022-11-01 Thread Simon Riggs
e no volunteers, maybe it is because the job of CFM is too big now and needs to be split. I can offer to be co-CFM, and think I can offer to be CFM for about 100 patches, but I don't have the time to do the whole thing myself. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Code checks for App Devs, using new options for transaction behavior

2022-10-31 Thread Simon Riggs
On Mon, 31 Oct 2022 at 12:22, Dilip Kumar wrote: > > On Mon, Oct 31, 2022 at 5:03 PM Dilip Kumar wrote: > > > > On Mon, Oct 31, 2022 at 4:23 PM Dilip Kumar wrote: > > > > > > On Sun, Oct 30, 2022 at 11:32 PM Simon Riggs > > > wrote: > > &g

Re: Code checks for App Devs, using new options for transaction behavior

2022-10-31 Thread Simon Riggs
the GUC. So its not a problem, but thanks for checking. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Code checks for App Devs, using new options for transaction behavior

2022-10-30 Thread Simon Riggs
On Fri, 28 Oct 2022 at 10:33, Simon Riggs wrote: > Thanks for the feedback, I will make all of those corrections in the > next version. New version attached. I've rolled 002-004 into one patch, but can split again as needed. -- Simon Riggshttp://www.Enter

Re: Code checks for App Devs, using new options for transaction behavior

2022-10-28 Thread Simon Riggs
On Fri, 28 Oct 2022 at 07:54, Erik Rijkers wrote: > > Op 27-10-2022 om 18:35 schreef Simon Riggs: > > On Thu, 27 Oct 2022 at 12:09, Simon Riggs > > wrote: > > > >> Comments please > > > > Update from patch tester results. > > > > > [

Re: Code checks for App Devs, using new options for transaction behavior

2022-10-27 Thread Simon Riggs
On Thu, 27 Oct 2022 at 12:09, Simon Riggs wrote: > Comments please Update from patch tester results. -- Simon Riggshttp://www.EnterpriseDB.com/ 002_nested_xacts.v7.patch Description: Binary data 001_psql_parse_only.v1.patch Description: Binary d

Re: Allow single table VACUUM in transaction block

2022-10-27 Thread Simon Riggs
On Thu, 27 Oct 2022 at 10:31, Simon Riggs wrote: > Tests, docs. The patch tester says that a pg_upgrade test is failing on Windows, but works for me. t/002_pg_upgrade.pl .. ok Anybody shed any light on that, much appreciated. -- Simon Riggshttp://www.EnterpriseDB.com/

Code checks for App Devs, using new options for transaction behavior

2022-10-27 Thread Simon Riggs
n purpose of these features is hopefully clear from these notes. 001 and 003 are fairly small patches, 002 is longer. Comments please -- Simon Riggshttp://www.EnterpriseDB.com/ 001_psql_parse_only.v1.patch Description: Binary data 002_nested_xacts.v7.patch Description: Binary da

Allow single table VACUUM in transaction block

2022-10-27 Thread Simon Riggs
. When in a xact block, we do not set PROC_IN_VACUUM, nor update datfrozenxid. Tests, docs. -- Simon Riggshttp://www.EnterpriseDB.com/ single_table_vacuum.v1.patch Description: Binary data

Re: New docs chapter on Transaction Management and related changes

2022-10-24 Thread Simon Riggs
actions created by the named savepoint, these will also be subcommitted. The point is that savepoints create subtransactions, but they are not the only way to create them, so we cannot equate savepoint and subtransaction completely. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: New docs chapter on Transaction Management and related changes

2022-10-14 Thread Simon Riggs
On Fri, 14 Oct 2022 at 08:55, Simon Riggs wrote: > In related changes, I've also done some major rewording of the RELEASE > SAVEPOINT command, since it was incorrectly described as having "no > other user visible behavior". A complex example is given to explain,

Re: New docs chapter on Transaction Management and related changes

2022-10-14 Thread Simon Riggs
xids are recorded in > > +the pg_multixact directory. > > + > > Hmm, ok. > > > + > > +The parent xid of each subxid is recorded in the > > +pg_subtrans directory. No entry is made for > > +top-level xids since they do not have a parent, nor is an entry made > > +for read-only subtransactions. > > + > > Maybe say "the immediate parent xid of each ...", or is it too obvious? +1 to all of those suggestions -- Simon Riggshttp://www.EnterpriseDB.com/

Re: New docs chapter on Transaction Management and related changes

2022-10-14 Thread Simon Riggs
ther user visible behavior". A complex example is given to explain, using the terminology established in the main patch. -- Simon Riggshttp://www.EnterpriseDB.com/ xact.patch-on-patch Description: Binary data doc-release-savepoint.v1.patch Description: Binary data

Re: Startup process on a hot standby crashes with an error "invalid memory alloc request size 1073741824" while replaying "Standby/LOCK" records

2022-10-10 Thread Simon Riggs
where > this helps significantly are at all mainstream, so I'm kind of leaning > to "patch HEAD only". It looks fine to eventually backpatch, since StandbyReleaseLockTree() was optimized to only be called when the transaction had actually done some AccessExclusiveLocks. So the performance loss is minor and isolated to the users of such locks, so I see no problems with it. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-09-26 Thread Simon Riggs
On Fri, 16 Sept 2022 at 13:20, Simon Riggs wrote: > > Thanks for the review. > > v10 attached v11 attached, corrected for recent commit 14ff44f80c09718d43d853363941457f5468cc03. -- Simon Riggshttp://www.EnterpriseDB.com/ 002_minimize_calls_to_SubTransSetPare

Re: Pruning never visible changes

2022-09-22 Thread Simon Riggs
On Thu, 22 Sept 2022 at 15:16, Alvaro Herrera wrote: > > On 2022-Sep-22, Simon Riggs wrote: > > > On Mon, 19 Sept 2022 at 00:16, Greg Stark wrote: > > > > VACUUM was willing to remove a committed-dead tuple immediately if it > > > was > > >

Re: Pruning never visible changes

2022-09-22 Thread Simon Riggs
On Mon, 19 Sept 2022 at 00:16, Greg Stark wrote: > > On Fri, 16 Sept 2022 at 10:27, Tom Lane wrote: > > > > Simon Riggs writes: > > > A user asked me whether we prune never visible changes, such as > > > BEGIN; > > > INSERT... > > > UPDATE.

Re: Hash index build performance tweak from sorting

2022-09-21 Thread Simon Riggs
On Wed, 21 Sept 2022 at 02:32, David Rowley wrote: > > On Tue, 2 Aug 2022 at 03:37, Simon Riggs wrote: > > Using the above test case, I'm getting a further 4-7% improvement on > > already committed code with the attached patch, which follows your > > proposal. > &

Re: Pruning never visible changes

2022-09-16 Thread Simon Riggs
On Fri, 16 Sept 2022 at 18:37, Tom Lane wrote: > > Simon Riggs writes: > > On Fri, 16 Sept 2022 at 15:26, Tom Lane wrote: > >> You cannot > >> do that, at least not without checking that the originating > >> transaction has no snapshots that could see

Re: Pruning never visible changes

2022-09-16 Thread Simon Riggs
On Fri, 16 Sept 2022 at 21:07, Laurenz Albe wrote: > > On Fri, 2022-09-16 at 10:26 -0400, Tom Lane wrote: > > Simon Riggs writes: > > > A user asked me whether we prune never visible changes, such as > > > BEGIN; > > > INSERT... > > > UPDATE.. (sam

Re: Slow standby snapshot

2022-09-16 Thread Simon Riggs
mail.gmail.com#379c1be7b8134ada5a574078d51b64c6 I've cleaned up the comments and used a #define for the constant. IMHO this is ready for commit. I will add it to the next CF. -- Simon Riggshttp://www.EnterpriseDB.com/ v8c-new-heuristic-to-compress-KnownAssignedXids.patch Description: Binary data

Re: Pruning never visible changes

2022-09-16 Thread Simon Riggs
On Fri, 16 Sept 2022 at 15:26, Tom Lane wrote: > > Simon Riggs writes: > > A user asked me whether we prune never visible changes, such as > > BEGIN; > > INSERT... > > UPDATE.. (same row) > > COMMIT; > > Didn't we just have this discussion in a

Pruning never visible changes

2022-09-16 Thread Simon Riggs
urs.) Please let me know what you think. -- Simon Riggshttp://www.EnterpriseDB.com/ never_visible.v1.patch Description: Binary data

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-09-16 Thread Simon Riggs
On Thu, 15 Sept 2022 at 12:36, Japin Li wrote: > > > On Thu, 15 Sep 2022 at 18:04, Simon Riggs > wrote: > > On Wed, 14 Sept 2022 at 15:21, Alvaro Herrera > > wrote: > >> > >> On 2022-Aug-30, Simon Riggs wrote: > >> > >> > 001_new_

  1   2   3   4   5   6   7   8   >