Error for WITH options on partitioned tables

2022-09-16 Thread Simon Riggs
partitions instead -- Simon Riggshttp://www.EnterpriseDB.com/ 001_test_options_with_partitioned_table.v1.patch Description: Binary data 002_better_error_for_options_on_partitioned.v1.patch Description: Binary data

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-09-15 Thread Simon Riggs
On Wed, 14 Sept 2022 at 15:21, Alvaro Herrera wrote: > > On 2022-Aug-30, Simon Riggs wrote: > > > 001_new_isolation_tests_for_subxids.v3.patch > > Adds new test cases to master without adding any new code, specifically > > addressing the two areas of code that are n

Re: New docs chapter on Transaction Management and related changes

2022-09-13 Thread Simon Riggs
On Sun, 11 Sept 2022 at 21:35, Robert Treat wrote: > > On Wed, Sep 7, 2022 at 8:05 AM Simon Riggs > wrote: > > > > On Tue, 6 Sept 2022 at 21:33, Justin Pryzby wrote: > > > > > > On Tue, Sep 06, 2022 at 04:16:02PM +0100, Simon Riggs wrote: > > &g

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-09-13 Thread Simon Riggs
On Tue, 13 Sept 2022 at 11:56, Simon Riggs wrote: > > On Tue, 6 Sept 2022 at 13:14, Simon Riggs > wrote: > > > I will update the patch, thanks for your scrutiny. > > I attach a diff showing what has changed between v8 and v9, and will > reattach a full set of new p

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-09-13 Thread Simon Riggs
On Tue, 6 Sept 2022 at 13:14, Simon Riggs wrote: > I will update the patch, thanks for your scrutiny. I attach a diff showing what has changed between v8 and v9, and will reattach a full set of new patches in the next post, so patchtester doesn't squeal. -- Simon Riggsh

Re: New docs chapter on Transaction Management and related changes

2022-09-09 Thread Simon Riggs
On Thu, 8 Sept 2022 at 08:42, Alvaro Herrera wrote: > > On 2022-Sep-06, Simon Riggs wrote: > > > On Tue, 6 Sept 2022 at 17:19, Erik Rijkers wrote: > > > > > > Op 06-09-2022 om 17:16 schreef Simon Riggs: > > > > New chapter on trans

Re: New docs chapter on Transaction Management and related changes

2022-09-07 Thread Simon Riggs
On Tue, 6 Sept 2022 at 21:33, Justin Pryzby wrote: > > On Tue, Sep 06, 2022 at 04:16:02PM +0100, Simon Riggs wrote: > > New chapter on transaction management, plus a few related changes. > > > > Markup and links are not polished yet, so please comment initially on >

Re: New docs chapter on Transaction Management and related changes

2022-09-06 Thread Simon Riggs
On Tue, 6 Sept 2022 at 17:19, Erik Rijkers wrote: > > Op 06-09-2022 om 17:16 schreef Simon Riggs: > > New chapter on transaction management, plus a few related changes. > > > > Markup and links are not polished yet, so please comment initially on > > the to

New docs chapter on Transaction Management and related changes

2022-09-06 Thread Simon Riggs
New chapter on transaction management, plus a few related changes. Markup and links are not polished yet, so please comment initially on the topics, descriptions and wording. -- Simon Riggshttp://www.EnterpriseDB.com/ xact_docs.v2.patch Description: Binary data

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-09-06 Thread Simon Riggs
On Tue, 6 Sept 2022 at 12:37, Dilip Kumar wrote: > > On Tue, Aug 30, 2022 at 10:16 PM Simon Riggs > wrote: > > > PFA two patches, replacing earlier work > > 001_new_isolation_tests_for_subxids.v3.patch > > 002_minimize_calls

Re: Smoothing the subtrans performance catastrophe

2022-09-06 Thread Simon Riggs
On Thu, 4 Aug 2022 at 13:11, Simon Riggs wrote: > I will post my patch, when complete, in a different thread. To avoid confusion, I will withdraw this patch from the CF, in favour of my other patch on a similar topic, Minimizing calls to SubTransSetParent() https://commitfest.postgresql.org

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-08-30 Thread Simon Riggs
smoothing the overflow case. Some discussion needed on this; there are various options. This combines the work originally posted here with another patch posted on the thread "Smoothing the subtrans performance catastrophe". I will do some performance testing also, but more welcome

Re: Hash index build performance tweak from sorting

2022-08-30 Thread Simon Riggs
On Fri, 5 Aug 2022 at 20:46, David Zhang wrote: > > On 2022-08-01 8:37 a.m., Simon Riggs wrote: > > Using the above test case, I'm getting a further 4-7% improvement on > > already committed code with the attached patch, which follows your > > proposal. > > I ran

Re: MERGE and parsing with prepared statements

2022-08-12 Thread Simon Riggs
ifference. By changing the examples so they are the same, the sentence at line 573 now makes no sense. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-08-10 Thread Simon Riggs
CSnapdhot() That is one way to resolve the issue, but not the only one. I can also change AssignTransactionId() to recursively register parent xids for all of a subxid's parents. I will add in a test case and resolve the dependency in my next patch. Thanks again. -- Simon Riggs

Blocking the use of TRIGGER privilege

2022-08-09 Thread Simon Riggs
ting usage of the trigger privilege would not be touched, only new usage. (No, this does not mean I want to ban triggers, only the trigger privilege). Thoughts? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-08-09 Thread Simon Riggs
to provide more detailed review thoughts, so please keep asking questions. -- Simon Riggshttp://www.EnterpriseDB.com/

SUBTRANS: Minimizing calls to SubTransSetParent()

2022-08-08 Thread Simon Riggs
On Thu, 4 Aug 2022 at 13:11, Simon Riggs wrote: > > On Wed, 3 Aug 2022 at 20:18, Andres Freund wrote: > > > I think we should consider redesigning subtrans more substantially - even > > with > > the changes you propose here, there's still plenty ways to hit

Re: Smoothing the subtrans performance catastrophe

2022-08-04 Thread Simon Riggs
On Wed, 3 Aug 2022 at 20:18, Andres Freund wrote: > On 2022-08-01 17:42:49 +0100, Simon Riggs wrote: > > The reason for the slowdown is clear: when we overflow we check every > > xid against subtrans, producing a large stream of lookups. Some > > previous hackers have tried

Re: Slow standby snapshot

2022-08-02 Thread Simon Riggs
o make that significantly better by starting each binary search from the next element, rather than the start of the array. Something like the attached might help, but we can probably make that cache conscious to improve things even more. -- Simon Riggshttp://www.Enterprise

Re: Slow standby snapshot

2022-08-02 Thread Simon Riggs
t in varying those numbers to see the effects. It did also occur to me that we might need a separate process to perform the compressions, which we might be able to give to WALWriter. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Dump/Restore of non-default PKs

2022-08-01 Thread Simon Riggs
OD hash > > should be able to avoid any future clashes. That seems to conflict with USING INDEX TABLESPACE. I've tried a few things but have not found anything yet. Any other ideas on syntax? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Hash index build performance tweak from sorting

2022-08-01 Thread Simon Riggs
On Fri, 29 Jul 2022 at 13:49, Simon Riggs wrote: > > On Thu, 28 Jul 2022 at 19:50, Tom Lane wrote: > > > > Simon Riggs writes: > > > Thanks for the nudge. New version attached. > > > > I also see a speed improvement from this > > --- > > D

Re: Slow standby snapshot

2022-07-29 Thread Simon Riggs
o keep the array from spreading out and impacting snapshot performance for SELECTs, yet not doing it so often that the startup process has a higher burden of work. -- Simon Riggshttp://www.EnterpriseDB.com/ subx_compress_knownassignedxids_more_often.v2.patch Description: Binary data

Re: Maximize page freezing

2022-07-29 Thread Simon Riggs
On Thu, 28 Jul 2022 at 14:55, Matthias van de Meent wrote: > > On Thu, 28 Jul 2022 at 15:36, Simon Riggs > wrote: > > > > Starting new thread with updated patch to avoid confusion, as > > mentioned by David Steele on the original thread: > > Original

Re: Maximize page freezing

2022-07-29 Thread Simon Riggs
page together appears in earlier patching that were posted. These were > things that didn't make it into Postgres 15. Yes, my patch from 2020 was never reviewed, which is why I was resubmitting here. > I should be able to post something in a couple of weeks. How do you see that a

Re: Hash index build performance tweak from sorting

2022-07-29 Thread Simon Riggs
On Thu, 28 Jul 2022 at 19:50, Tom Lane wrote: > > Simon Riggs writes: > > Thanks for the nudge. New version attached. > > I also see a speed improvement from this, so pushed (after minor comment > editing). Thanks > I notice though that if I feed it random data, > &

Maximize page freezing

2022-07-28 Thread Simon Riggs
Starting new thread with updated patch to avoid confusion, as mentioned by David Steele on the original thread: Original messageid: 20201118020418.GA13408@alvherre.pgsql On Wed, 18 Nov 2020 at 02:04, Alvaro Herrera wrote: > On 2020-Nov-17, Simon Riggs wrote: > > > As an additional

Re: Hash index build performance tweak from sorting

2022-07-28 Thread Simon Riggs
On Wed, 27 Jul 2022 at 19:22, Tom Lane wrote: > > Simon Riggs writes: > > [ hash_sort_by_hash.v2.patch ] > > The cfbot says this no longer applies --- probably sideswiped by > Korotkov's sorting-related commits last night. Thanks for the nudge. New version attac

Re: [PATCH] Compression dictionaries for JSONB

2022-07-27 Thread Simon Riggs
ictionary in the catalog prevent that dictionary from being used within the toaster? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2022-07-26 Thread Simon Riggs
's have a democratic vote on moving this forwards: +1 from me. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Max compact as an FSM strategy

2022-07-26 Thread Simon Riggs
On Tue, 26 Jul 2022 at 11:02, Dilip Kumar wrote: > > On Tue, Jul 26, 2022 at 3:04 PM Simon Riggs > wrote: > > > > A long time ago, Tom Lane came up with the idea that when tables get > > bloated, tables might be allowed to shrink down again in size > > naturally

Max compact as an FSM strategy

2022-07-26 Thread Simon Riggs
. +*/ +} FreeSpaceStrategy; All we need is a simple heuristic to allow us to choose between various strategies. Your input is welcome! Please read the short patch. -- Simon Riggshttp://www.EnterpriseDB.com/ freespace_strategy.v2.patch Description: Binary data

Re: StrategyAM for IndexAMs

2022-07-22 Thread Simon Riggs
On Fri, 22 Jul 2022 at 10:23, Matthias van de Meent wrote: > > On Tue, 19 Jul 2022 at 18:56, Simon Riggs > wrote: > > > > I'm preparing the way for a later patch that would allow unique hash > > indexes to be primary keys. There are various parts to the problem. I &g

Re: Make name optional in CREATE STATISTICS

2022-07-21 Thread Simon Riggs
On Thu, 21 Jul 2022 at 15:12, Dean Rasheed wrote: > > On Wed, 20 Jul 2022 at 12:01, Matthias van de Meent > wrote: > > > > On Wed, 13 Jul 2022 at 08:07, Simon Riggs > > wrote: > > > > > > > + *CREATE STATISTICS [ [IF NOT EX

StrategyAM for IndexAMs

2022-07-19 Thread Simon Riggs
e checked. So before I do that, it seems sensible to agree the approach. (Obviously, there are hundreds of places where BTEqualStrategyNumber is hardcoded, and this doesn't change that at all, in case that wasn't clear). Comments welcome on this still WIP patch. -- Simon Riggs

Re: Allowing REINDEX to have an optional name

2022-07-17 Thread Simon Riggs
On Sun, 17 Jul 2022 at 07:19, Michael Paquier wrote: > > On Fri, Jul 15, 2022 at 06:21:22PM +0100, Simon Riggs wrote: > > That's fixed it on the CFbot. Over to you, Michael. Thanks. > > Sure. I have looked over that, and this looks fine overall. I have > ma

Re: Allowing REINDEX to have an optional name

2022-07-15 Thread Simon Riggs
On Fri, 15 Jul 2022 at 12:25, Simon Riggs wrote: > > On Mon, 4 Jul 2022 at 08:00, Michael Paquier wrote: > > > > On Sun, Jul 03, 2022 at 05:41:31PM -0400, Tom Lane wrote: > > > This is marked as Ready for Committer, but that seems unduly > > > optimistic. &g

Re: Allowing REINDEX to have an optional name

2022-07-15 Thread Simon Riggs
rd recommendation on macOS with SIP on is to always do "make > install" before "make check". Thanks, will investigate. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Allowing REINDEX to have an optional name

2022-07-15 Thread Simon Riggs
> get_database_name(), something that there is no need for as long as > the routine is called once in ReindexMultipleTables(). OK, let me repost the new patch and see if CFbot likes that better. This includes Michael's other requested changes for reindexdb. -- Simon Riggshttp://

Re: Allowing REINDEX to have an optional name

2022-07-15 Thread Simon Riggs
On Fri, 15 Jul 2022 at 12:15, Simon Riggs wrote: > > On Fri, 15 Jul 2022 at 09:12, Alvaro Herrera wrote: > > > > On 2022-Jul-15, Simon Riggs wrote: > > > > > On Fri, 15 Jul 2022 at 04:44, Michael Paquier wrote: > > > > > > This patch has been

Re: Allowing REINDEX to have an optional name

2022-07-15 Thread Simon Riggs
On Fri, 15 Jul 2022 at 09:12, Alvaro Herrera wrote: > > On 2022-Jul-15, Simon Riggs wrote: > > > On Fri, 15 Jul 2022 at 04:44, Michael Paquier wrote: > > > > This patch has been marked as waiting for a review, however the CF bot > > > is completely red

Re: Allowing REINDEX to have an optional name

2022-07-15 Thread Simon Riggs
itfest.cputube.org/simon-riggs.html Thanks for showing me that, it is very helpful. > This patch has been marked as waiting for a review, however the CF bot > is completely red: Yes, it is failing, but so is current HEAD, with some kind of libpq pipelining error. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Make name optional in CREATE STATISTICS

2022-07-13 Thread Simon Riggs
On Thu, 7 Jul 2022 at 11:58, Matthias van de Meent wrote: > > On Thu, 7 Jul 2022 at 12:55, Simon Riggs wrote: > > There are various other ways of doing this and, yes, we could refactor > > other parts of the grammar to make this work. There is a specific > > guidelin

Re: Make name optional in CREATE STATISTICS

2022-07-07 Thread Simon Riggs
On Wed, 6 Jul 2022 at 19:35, Matthias van de Meent wrote: > > On Sun, 15 May 2022 at 14:20, Simon Riggs > wrote: > > > > Currently, CREATE STATS requires you to think of a name for each stats > > object, which is fairly painful, so users would prefer an >

Re: making relfilenodes 56 bits

2022-06-30 Thread Simon Riggs
On Thu, 30 Jun 2022 at 03:43, Thomas Munro wrote: > > On Thu, Jun 30, 2022 at 12:41 AM Simon Riggs > wrote: > > The reason to mention this now is that it would give more space than > > 56bit limit being suggested here. > > Isn't 2^56 enough, though? For me, ye

Re: Allowing REINDEX to have an optional name

2022-06-29 Thread Simon Riggs
On Wed, 29 Jun 2022 at 05:35, Michael Paquier wrote: > > On Tue, Jun 28, 2022 at 11:02:25AM +0100, Simon Riggs wrote: > > Attached patch is tested, documented and imho ready to be committed, > > so I will mark it so in CFapp. Thanks for the review Michael. > Th

Re: making relfilenodes 56 bits

2022-06-29 Thread Simon Riggs
ld cache nicely. For convenience, I imagine that the mapping could be included in WAL in or near the checkpoint record, to ensure that the mapping was available in all backups. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: making relfilenodes 56 bits

2022-06-28 Thread Simon Riggs
RelFileNumber, without restriction, and probably some spare bytes on the ForkNumber, if we needed them later. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Allowing REINDEX to have an optional name

2022-06-28 Thread Simon Riggs
On Tue, 28 Jun 2022 at 08:29, Simon Riggs wrote: > > On Thu, 2 Jun 2022 at 01:02, Michael Paquier wrote: > > > > On Tue, May 31, 2022 at 02:30:32PM +0200, Alvaro Herrera wrote: > > > I was thinking the opposite: REINDEX DATABASE with or without a database >

Re: Allowing REINDEX to have an optional name

2022-06-28 Thread Simon Riggs
catalogs, doing the same > thing as REINDEX DATABASE today? A key point is that REINDEX SYSTEM has problems, so should be avoided. Hence, including both database and system together in a new command would not be great idea, at this time. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Race condition in TransactionIdIsInProgress

2022-06-25 Thread Simon Riggs
new with this > patch, though, it could happen with the single-item cache in transam.c, too. Yes, as a separate patch only in PG15+, imho. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: O(n) tasks cause lengthy startups and checkpoints

2022-06-24 Thread Simon Riggs
On Thu, 23 Jun 2022 at 18:15, Nathan Bossart wrote: > I'm grateful for the discussion in this thread so far, but I'm not seeing a > clear path forward. +1 to add the new auxiliary process. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: allow specifying action when standby encounters incompatible parameter settings

2022-06-24 Thread Simon Riggs
On Thu, 23 Jun 2022 at 18:45, Nathan Bossart wrote: > > Thanks for chiming in. > > On Thu, Jun 23, 2022 at 04:19:45PM +0100, Simon Riggs wrote: > > I don't understand why you need this patch at all. > > > > Since you have automation, you can use that layer to

Re: O(n) tasks cause lengthy startups and checkpoints

2022-06-23 Thread Simon Riggs
On Thu, 23 Jun 2022 at 14:46, Robert Haas wrote: > > On Thu, Jun 23, 2022 at 7:58 AM Simon Riggs > wrote: > > Having a central cleanup process makes a lot of sense. There is a long > > list of potential tasks for such a process. My understanding is that > > autovacu

Re: [PATCH] Compression dictionaries for JSONB

2022-06-23 Thread Simon Riggs
s for those future enhancements. i.e. how will we do ALTER TABLE ... UPGRADE DICTIONARY without causing a table rewrite? * Does the order of entries in the dictionary allow us to express a priority? i.e. to allow Huffman coding. Thanks for your efforts - this is a very important patch. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: allow specifying action when standby encounters incompatible parameter settings

2022-06-23 Thread Simon Riggs
is allow you to implement things in a lazy way that causes a loss of availability for users. I don't think that is of benefit to PostgreSQL users, so -1 from me, on this patch (only), sorry about that. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: doc: Clarify Savepoint Behavior

2022-06-23 Thread Simon Riggs
int name duplicated earlier defined savepoints then those earlier savepoints can only be released if multiple ROLLBACK TO SAVEPOINT commands are issued with the same name, as shown in the following example." Also, I would just call the savepoint "s" in the example, to declutter it. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: O(n) tasks cause lengthy startups and checkpoints

2022-06-23 Thread Simon Riggs
good, if that is possible. That can be done better over time. This point does not block the higher level goal of better availability asap, so I support Nathan's overall proposals. -- Simon Riggshttp://www.EnterpriseDB.com/

Make name optional in CREATE STATISTICS

2022-05-15 Thread Simon Riggs
is simple, but that's exactly what users do anyway, so it is not too bad. Tests, docs included. -- Simon Riggshttp://www.EnterpriseDB.com/ create_stats_opt_name.v3.patch Description: Binary data

Re: Comments on Custom RMGRs

2022-05-13 Thread Simon Riggs
On Fri, 13 May 2022 at 00:42, Andres Freund wrote: > > On 2022-05-12 22:26:51 +0100, Simon Riggs wrote: > > On Thu, 12 May 2022 at 04:40, Andres Freund wrote: > > > I'm not happy with the idea of random code being executed in the middle of > > > CheckPointGut

Re: Comments on Custom RMGRs

2022-05-13 Thread Simon Riggs
On Fri, 13 May 2022 at 05:13, Jeff Davis wrote: > > On Thu, 2022-05-12 at 22:26 +0100, Simon Riggs wrote: > > I see multiple uses for the rm_checkpoint() point proposed and I've > > been asked multiple times for a checkpoint hook. > > Can you elaborate and/or link to a

Re: Comments on Custom RMGRs

2022-05-12 Thread Simon Riggs
On Thu, 12 May 2022 at 04:40, Andres Freund wrote: > > On 2022-05-11 09:39:48 -0700, Jeff Davis wrote: > > On Wed, 2022-05-11 at 15:24 +0100, Simon Riggs wrote: > > > [PATCH: rmgr_001.v1.patch] > > > > > > [PATCH: rmgr_002.v1.patch] > > > > T

Re: Allowing REINDEX to have an optional name

2022-05-11 Thread Simon Riggs
BASE the same as REINDEX (SKIP_SYSTEM_CATALOGS OFF) DATABASE then it is just dangerous and annoying, i.e. a POLA violation. The point of this was a usability improvement, not just new syntax. -- Simon Riggshttp://www.EnterpriseDB.com/

Comments on Custom RMGRs

2022-05-11 Thread Simon Riggs
ng limited in scope. So I think we also need an rm_checkpoint() call - which would be a no-op for existing rmgrs. [PATCH: rmgr_003.v1.patch] The above turns out to be fairly simple, but extends the API to something truly flexible. Please let me know what you think? -- Simon Riggs

Re: Allowing REINDEX to have an optional name

2022-05-10 Thread Simon Riggs
On Tue, 10 May 2022 at 14:47, Ashutosh Bapat wrote: > > On Tue, May 10, 2022 at 2:43 PM Simon Riggs > wrote: > > > > A minor issue, and patch. > > > > REINDEX DATABASE currently requires you to write REINDEX DATABASE > > dbname, which makes this

Re: Hash index build performance tweak from sorting

2022-05-10 Thread Simon Riggs
based on the buckets they belong to. Below masks > * are used in _hash_hashkey2bucket to determine the bucket of given hash > * key. > */ Addressed in new patch, v2. On Wed, 4 May 2022 at 11:27, Amit Kapila wrote: > > So, we should go with this unless someone else sees any flaw h

Allowing REINDEX to have an optional name

2022-05-10 Thread Simon Riggs
skips catalog tables, as a way of avoiding the known deadlocks. Patch also adds a test. -- Simon Riggshttp://www.EnterpriseDB.com/ reindex_not_require_database_name.v2.patch Description: Binary data

Re: Hash index build performance tweak from sorting

2022-05-02 Thread Simon Riggs
On Sat, 30 Apr 2022 at 12:12, Amit Kapila wrote: > > On Tue, Apr 19, 2022 at 3:05 AM Simon Riggs > wrote: > > > > Hash index pages are stored in sorted order, but we don't prepare the > > data correctly. > > > > We sort the data as the first step of a hash

Re: Dump/Restore of non-default PKs

2022-04-21 Thread Simon Riggs
we are clear this is not about hash indexes, this is about using ANY kind of index (i.e. any index access method, extension or otherwise) to enforce a constraint. Another idea might be to allow some kind of statement embedding... so we don't need to constantly fiddle with ALTER TABLE ALTER TABLE

Re: Dump/Restore of non-default PKs

2022-04-20 Thread Simon Riggs
be fixed before we can progress further on this thread. Will look into it. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Dump/Restore of non-default PKs

2022-04-19 Thread Simon Riggs
On Mon, 18 Apr 2022 at 22:05, Simon Riggs wrote: > > On Mon, 18 Apr 2022 at 21:48, Tom Lane wrote: > > > > "David G. Johnston" writes: > > > On Mon, Apr 18, 2022 at 1:00 PM Simon Riggs > > > wrote: > > >> I propose that we change p

Hash index build performance tweak from sorting

2022-04-18 Thread Simon Riggs
. -- Simon Riggshttp://www.EnterpriseDB.com/ hash_sort_by_hash.v1.patch Description: Binary data hashtest.sql Description: Binary data

Re: Dump/Restore of non-default PKs

2022-04-18 Thread Simon Riggs
On Mon, 18 Apr 2022 at 21:48, Tom Lane wrote: > > "David G. Johnston" writes: > > On Mon, Apr 18, 2022 at 1:00 PM Simon Riggs > > wrote: > >> I propose that we change pg_dump so that when it creates a PK it does > >> so in 2 commands: > >>

Dump/Restore of non-default PKs

2022-04-18 Thread Simon Riggs
h defaults Doing this as 2 steps instead of one doesn't add any more time because (2) is just a metadata-only change, not an index build. Any objections to a patch to implement this thought? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: JSON docs: RFC7159 is now superceded

2022-04-13 Thread Simon Riggs
On Wed, 13 Apr 2022 at 14:53, Tom Lane wrote: > > Simon Riggs writes: > > Minor doc patch to replace with latest RFC number > > Hmm, I'm a bit disinclined to claim compliance with a new RFC > sight unseen. What were the changes? I checked... so I should have mention

JSON docs: RFC7159 is now superceded

2022-04-13 Thread Simon Riggs
Minor doc patch to replace with latest RFC number Intended for PG15 -- Simon Riggshttp://www.EnterpriseDB.com/ json_docs_rfc8259.v1.patch Description: Binary data

Re: WIP: WAL prefetch (another approach)

2022-04-12 Thread Simon Riggs
On Tue, 12 Apr 2022 at 16:41, Tomas Vondra wrote: > > On 4/12/22 15:58, Simon Riggs wrote: > > On Thu, 7 Apr 2022 at 08:46, Thomas Munro wrote: > > > >> With that... I've finally pushed the 0002 patch and will be watching > >> the build farm. > > > &

Re: WIP: WAL prefetch (another approach)

2022-04-12 Thread Simon Riggs
ftware that allows full_page_writes to be safely disabled. Perhaps something has been released recently that allows this? I think we have substantial documentation about safety of other settings, so we should carefully document things here also. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: suboverflowed subtransactions concurrency performance optimize

2022-04-10 Thread Simon Riggs
On Thu, 7 Apr 2022 at 00:36, Michael Paquier wrote: > > On Mon, Mar 07, 2022 at 10:17:41PM +0800, Julien Rouhaud wrote: > > On Mon, Mar 07, 2022 at 01:27:40PM +, Simon Riggs wrote: > >> The patch was posted because TransactionLogFetch() has a cache, yet > >>

Re: Documenting when to retry on serialization failure

2022-03-24 Thread Simon Riggs
On Thu, 24 Mar 2022 at 16:29, Tom Lane wrote: > > Simon Riggs writes: > > On Thu, 24 Mar 2022 at 14:56, Tom Lane wrote: > >> Um, what's that got to do with it? The example in > >> read-write-unique-4.spec involves only a single pkey constraint. > > > Yes,

Re: Parameter for planner estimate of recursive queries

2022-03-24 Thread Simon Riggs
On Thu, 24 Mar 2022 at 15:48, Tom Lane wrote: > > Simon Riggs writes: > > On Wed, 23 Mar 2022 at 20:25, Tom Lane wrote: > >> Do you have any objection if I rename the GUC to > >> recursive_worktable_factor? That seems a bit clearer as to what > >> it

Re: Reducing power consumption on idle servers

2022-03-24 Thread Simon Riggs
On Thu, 24 Mar 2022 at 15:39, Robert Haas wrote: > > On Thu, Mar 24, 2022 at 6:59 AM Simon Riggs > wrote: > > The proposals of this patch are the following, each of which can be > > independently accepted/rejected: > > 1. fix the sleep pattern of bgwriter, w

Re: Documenting when to retry on serialization failure

2022-03-24 Thread Simon Riggs
On Thu, 24 Mar 2022 at 14:56, Tom Lane wrote: > > Simon Riggs writes: > > OK, I see what you mean. There are 2 types of transaction, one that > > reads inside the transaction, one that decides what value to use some > > other way. > > > So now we have 2 cases,

Re: Documenting when to retry on serialization failure

2022-03-24 Thread Simon Riggs
On Thu, 24 Mar 2022 at 14:05, Tom Lane wrote: > > Thomas Munro writes: > > On Thu, Mar 24, 2022 at 11:44 PM Simon Riggs > > wrote: > >> The unique violation thing is worryingly general. Do we know enough to > >> say that this is thought to occur only with a)

Re: Documenting when to retry on serialization failure

2022-03-24 Thread Simon Riggs
On Thu, 24 Mar 2022 at 11:01, Thomas Munro wrote: > > On Thu, Mar 24, 2022 at 11:44 PM Simon Riggs > wrote: > > The unique violation thing is worryingly general. Do we know enough to > > say that this is thought to occur only with a) multiple unique > > constraint

Re: Reducing power consumption on idle servers

2022-03-24 Thread Simon Riggs
we don't forget one later 4. provide a design pattern for background worker extensions to follow, so as to encourage powersaving Please don't reject the whole patch because you disagree with part (3), it is not that important. Thanks -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Documenting when to retry on serialization failure

2022-03-24 Thread Simon Riggs
On Wed, 23 Mar 2022 at 19:50, Tom Lane wrote: > > Simon Riggs writes: > > I've tried to sum up the various points from everybody into this doc > > patch. Thanks all for replies. > > This seemed rather badly in need of copy-editing. How do you > like the attached

Re: Parameter for planner estimate of recursive queries

2022-03-24 Thread Simon Riggs
On Wed, 23 Mar 2022 at 20:25, Tom Lane wrote: > > Simon Riggs writes: > >> [New patch version pending] > > Do you have any objection if I rename the GUC to > recursive_worktable_factor? That seems a bit clearer as to what > it does, and it leaves more room for o

Re: Parameter for planner estimate of recursive queries

2022-03-23 Thread Simon Riggs
On Wed, 23 Mar 2022 at 18:20, Simon Riggs wrote: > [New patch version pending] -- Simon Riggshttp://www.EnterpriseDB.com/ recursive_worktable_estimate.v3.patch Description: Binary data

Re: Reducing power consumption on idle servers

2022-03-23 Thread Simon Riggs
On Tue, 22 Mar 2022 at 00:54, Andres Freund wrote: > > On 2022-02-21 21:04:19 +, Simon Riggs wrote: > > On Mon, 21 Feb 2022 at 16:49, Chapman Flack wrote: > > > > > Shouldn't the comment be "with work_done=false" ? > > > > Good c

Re: Parameter for planner estimate of recursive queries

2022-03-23 Thread Simon Riggs
nd would be. Agreed, makes sense. > * The proposed docs claim that a smaller setting works by biasing > the planner towards fast-start plans, but I don't think I believe > that explanation. I'd venture that we want text more along the > lines of "This may help the planne

Re: Parameter for planner estimate of recursive queries

2022-03-23 Thread Simon Riggs
On Tue, 22 Mar 2022 at 00:04, Andres Freund wrote: > On 2022-03-10 17:42:14 +0000, Simon Riggs wrote: > > Shall I set this as Ready For Committer? > > Currently this CF entry fails on cfbot: > https://cirrus-ci.com/task/4531771134967808?logs=test_world#L1158 > > [16:27

Re: Reducing power consumption on idle servers

2022-03-10 Thread Simon Riggs
0s would decrease this. I don't see much difference between power consumption with timeouts of 60s and 300s. In the latest patch, I chose 300s. Does anyone have an opinion on the value here? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Parameter for planner estimate of recursive queries

2022-03-10 Thread Simon Riggs
gt; us realistically doing a lot better with any reasonable amount of > work. Shall I set this as Ready For Committer? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: support for MERGE

2022-03-10 Thread Simon Riggs
current inserts are checked by merge-insert-update.spec, which correctly raises an ERROR in this case, as documented. Various cases were not tested in the patch - additional patch attached, but nothing surprising there. ExecInsert() does not return from such an ERROR, so the code fragment appears correct to me. -- Simon Riggshttp://www.EnterpriseDB.com/ new_insert_tests_for_merge.patch Description: Binary data

Re: Reducing power consumption on idle servers

2022-03-09 Thread Simon Riggs
gone the mechanism would be to request promotion via pg_ctl promote (the new API), which then creates the trigger file and sends a signal. So in both APIs the trigger file is still used. In this patch, if you create the trigger file directly/explicitly/yourself (the old API) then it will

Re: suboverflowed subtransactions concurrency performance optimize

2022-03-07 Thread Simon Riggs
On Mon, 7 Mar 2022 at 09:49, Julien Rouhaud wrote: > > Hi, > > On Mon, Jan 17, 2022 at 01:44:02PM +, Simon Riggs wrote: > > > > Re-attached, so that the CFapp isn't confused between the multiple > > patches on this thread. > > Thanks a lot for work

Re: BufferAlloc: don't take two simultaneous locks

2022-02-25 Thread Simon Riggs
'm missing > something. Sounds reasonable. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: BufferAlloc: don't take two simultaneous locks

2022-02-24 Thread Simon Riggs
hash from 32 to maybe 8, as originally speculated by Robert in 2016: https://www.postgresql.org/message-id/CA%2BTgmoZkg-04rcNRURt%3DjAG0Cs5oPyB-qKxH4wqX09e-oXy-nw%40mail.gmail.com since the freelists will be much less contended with the above approach It would be useful to see performance with a higher number of connections, >400. -- Simon Riggshttp://www.EnterpriseDB.com/

<    1   2   3   4   5   6   7   >