Re: Parallel safety docs for CTEs

2025-03-12 Thread James Coleman
On Tue, Nov 19, 2024 at 2:16 PM James Coleman wrote: > > Hello, > > A colleague noticed today that the docs still say that "Scans of > common table expressions (CTEs)" are "always parallel restricted". > > While I think that strictly remains true at

Idea: lock_timeout scoped by lock types

2025-01-29 Thread James Coleman
vel is something that would be a reasonable solution to this problem, or if folks think there's a better way to solve the problem. Regards, James Coleman

Re: Question about behavior of deletes with REPLICA IDENTITY NOTHING

2024-12-17 Thread James Coleman
iO3dfUnVmBs4M-4aUX_zHmPN72ELE7c_8qAO_toPmA%40mail.gmail.com > > Kind Regards, > Peter Smith. > Fujitsu Australia Yes, I would appreciate it moving forward. Did you want to create the CF entry or should I? Regards, James Coleman

Parallel safety docs for CTEs

2024-11-19 Thread James Coleman
since the change to default to trying to inline CTEs rather than defaulting to materializing them. Attached is a patch to slightly modify the language; would be happy to hear suggestions on a better way to improve this. Regards, James Coleman v1-0001-Update-parallel-safety-docs-for-CTE-optimizat

Re: Seq scan instead of index scan querying single row from primary key on large table

2024-07-18 Thread James Coleman
On Thu, Jul 18, 2024 at 2:38 PM Tom Lane wrote: > > James Coleman writes: > > The plan generated by the planner changed suddenly one morning this > > week, and in a very surprising way: the innermost scan (of "objects") > > started choosing a seq scan, des

Seq scan instead of index scan querying single row from primary key on large table

2024-07-18 Thread James Coleman
7;s true I think the early return cost multiplication of the LIMIT is being applied very naively on the seq scan node. Or perhaps the issue is that the startup cost for a single tuple on a seq scan like this shouldn't really have a startup cost of 0 -- that cost is presumably for tuples being returned _without_ having applied the filter. That seems slightly odd to me, because the cost of getting the first row out of that node -- in my naive view thinking about it for all of 5 seconds -- should be calculated based on applying the filter (and thus the likelihood that that filter matches right away). If we did that then this path would never win. But that 0.00 startup cost for the seq scan with a filter shows up in PG14 and PG11 also, not just PG16, so that's not something that's changed. To recap: the estimation of rows is correct, the estimated high (total) cost of the seq scan is correct, but the seq scan is chosen over the index scan anyway for a plan that returns a single "random" row based on the primary key. Am I right to be surprised here? James Coleman

Re: Should we document how column DEFAULT expressions work?

2024-07-01 Thread James Coleman
re in a kind of insidious way: the "bare" function call in DEFAULT is *not* executed as part of the query for DDL like it is with other queries. Hope this helps explain things. James Coleman

Re: Should we document how column DEFAULT expressions work?

2024-06-25 Thread James Coleman
On Tue, Jun 25, 2024 at 4:59 PM Tom Lane wrote: > > James Coleman writes: > > It's possible I'm the only one who's been in this situation, but I've > > multiple times found myself explaining to a user how column DEFAULT > > expressions work: namel

Should we document how column DEFAULT expressions work?

2024-06-25 Thread James Coleman
possible I'm missing something. Thanks, James Coleman

Re: Fix grammar oddities in comments

2024-06-10 Thread James Coleman
On Wed, Jun 5, 2024 at 5:34 AM David Rowley wrote: > > On Sun, 2 Jun 2024 at 10:08, James Coleman wrote: > > See attached for a small patch fixing some typos and grammatical > > errors in a couple of comments. > > Thanks. I pushed this after messing with the comm

Fix grammar oddities in comments

2024-06-01 Thread James Coleman
of the comment opting to simply resolve the clear mistakes in the wording here. Regards, James Coleman v1-0001-Fix-comment-grammar-oddities.patch Description: Binary data

Re: Add last_commit_lsn to pg_stat_database

2024-05-28 Thread James Coleman
RecordTransactionCommitPrepared() for this to work, and, indeed, adding some logging to verify, the value of XactLastRecEnd we'd use to update XactLastCommitEnd is the same at the end of both of those functions during COMMIT PREPARED. I'd like to have someone weigh in on whether relying on

Re: Add last_commit_lsn to pg_stat_database

2024-05-28 Thread James Coleman
nd 2. last_commit_lsn is not advancing, and 3. pg_current_wal_lsn() has advanced a lot then you can probably infer that there's a large amount of data that simply cannot be completed by the subscriber, and so there's no "real" delay. It also gives you an idea of how much data you will need to churn through (even if not replicated) once the transaction commits. Certainly understanding the data here will be simplest in the case where 1.) there's a single database and 2.) all tables are in the replication set, but I don't think the value is limited to that situation either. Regards, James Coleman

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-28 Thread James Coleman
erve to highlight patches that simply haven't had any review at all. I'd like to add a feature to the CF app that shows me my current patches by status, and I'd also like to have the option to have the CF app notify me when someone changes the status (I've noticed before that often a status gets changed without notification on list, and then I get surprised months later when it's stuck in "waiting on author"). Do either/both of those seem reasonable to add? Regards, James Coleman

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-28 Thread James Coleman
t;, I'd far prefer to have that outcome then some automated process closing out my submission without my input when, as far as I can tell, the real problem is not my lack of activity by the required reviewers simply not looking at it. So I'm genuinely confused by you say "As long as this is not a surprise ending, I don't see the issue.". Perhaps we're imagining something different here reading between the lines? Regards, James Coleman

Re: Teach predtest about IS [NOT] proofs

2024-04-05 Thread James Coleman
On Mon, Apr 1, 2024 at 8:06 AM James Coleman wrote: > > On Mon, Mar 25, 2024 at 5:53 PM Tom Lane wrote: > > > > James Coleman writes: > > > [ v6 patchset ] > > > > I went ahead and committed 0001 after one more round of review > > > >

Re: Teach predtest about IS [NOT] proofs

2024-04-01 Thread James Coleman
On Mon, Mar 25, 2024 at 5:53 PM Tom Lane wrote: > > James Coleman writes: > > [ v6 patchset ] > > I went ahead and committed 0001 after one more round of review > > statements; my bad). I also added the changes in test_predtest.c from > 0002. I attach a rebased versi

Re: Teach predtest about IS [NOT] proofs

2024-04-01 Thread James Coleman
tant, but perhaps future readers of the archives will > be confused. I was wondering myself :) so thanks for clarifying. Regards, James Coleman

Re: [DOCS] HOT - correct claim about indexes not referencing old line pointers

2024-03-23 Thread James Coleman
On Thu, Mar 21, 2024 at 1:09 PM Robert Haas wrote: > > On Thu, Mar 14, 2024 at 9:07 PM James Coleman wrote: > > If the goal here is the most minimal patch possible, then please > > commit what you proposed. I am interested in improving the document > > further, but I d

Re: [DOCS] HOT - correct claim about indexes not referencing old line pointers

2024-03-23 Thread James Coleman
On Wed, Mar 20, 2024 at 2:15 PM Robert Haas wrote: > > On Thu, Mar 14, 2024 at 9:07 PM James Coleman wrote: > > Obviously I have reasons for the other changes I made: for example, > > "no longer visible" improves the correctness, since being an old > > version i

Re: [DOCS] HOT - correct claim about indexes not referencing old line pointers

2024-03-14 Thread James Coleman
On Thu, Mar 14, 2024 at 10:28 AM Robert Haas wrote: > > On Wed, Oct 4, 2023 at 9:12 PM James Coleman wrote: > > All right, attached is a v3 which attempts to fix the wrong > > information with an economy of words. I may at some point submit a > > separate patch tha

Re: RFC: Logging plan of the running query

2024-03-02 Thread James Coleman
> > If this is correctly implemented, the overhead in the case where the > feature isn't used should be essentially zero, I believe. If I can rephrase this idea: it's basically "delay this interrupt until inline to the next ExecProcNode execution". That seems pretty promising to me as well. Regards, James Coleman

Re: RFC: Logging plan of the running query

2024-02-24 Thread James Coleman
On Fri, Feb 23, 2024 at 10:23 AM Robert Haas wrote: > > On Fri, Feb 23, 2024 at 7:50 PM Julien Rouhaud wrote: > > On Fri, Feb 23, 2024 at 10:22:32AM +0530, Robert Haas wrote: > > > On Thu, Feb 22, 2024 at 6:25 AM James Coleman wrote: > > > > This is potent

Re: RFC: Logging plan of the running query

2024-02-21 Thread James Coleman
ewhere. That's not likely to ever be as stable as we want > PostgreSQL to be. This is potentially a bit of a wild idea, but I wonder if having some kind of argument to CHECK_FOR_INTERRUPTS() signifying we're in "normal" as opposed to "critical" (using that word differently than the existing critical sections) would be worth it. Regards, James Coleman

Re: Question about behavior of deletes with REPLICA IDENTITY NOTHING

2024-02-08 Thread James Coleman
> NOTHING > > Records no information about the old row. This is equivalent to having > no replica identity. This is the default for system tables. This is the simplest change, and it does solve the confusion, so I'd be happy with it also. The other proposals have the benefit of having all the information necessary on the publications page rather than requiring the user to refer to the ALTER TABLE REPLICA IDENTITY page to understand what's meant. Regards, James Coleman

Re: Question about behavior of deletes with REPLICA IDENTITY NOTHING

2024-02-08 Thread James Coleman
or index that doesn't exist) is added ... I think that would work also. I was reading the initial suggestion as "(or with replica identity behavior the same as..." as defining what "without a replica identity" meant, which would avoid the confusion. But your proposal is more explicit and more succinct, so I think it's the better option of the two. Regards, James Coleman

Re: Question about behavior of deletes with REPLICA IDENTITY NOTHING

2024-02-07 Thread James Coleman
On Wed, Feb 7, 2024 at 6:04 PM Peter Smith wrote: > > On Thu, Feb 8, 2024 at 9:04 AM James Coleman wrote: > > > > On Wed, Feb 7, 2024 at 3:22 PM Laurenz Albe > > wrote: > > > > > > On Wed, 2024-02-07 at 15:12 -0500, James Coleman wrote: > >

Re: Question about behavior of deletes with REPLICA IDENTITY NOTHING

2024-02-07 Thread James Coleman
On Wed, Feb 7, 2024 at 3:22 PM Laurenz Albe wrote: > > On Wed, 2024-02-07 at 15:12 -0500, James Coleman wrote: > > We recently noticed some behavior that seems reasonable but also > > surprised our engineers based on the docs. > > > > If we have this setup

Question about behavior of deletes with REPLICA IDENTITY NOTHING

2024-02-07 Thread James Coleman
ble. I'm wondering if this might be a surprise to anyone else, and if so, is there a minor docs tweak that might avoid the confusion? Thanks, James Coleman 1: https://www.postgresql.org/docs/current/sql-altertable.html#SQL-ALTERTABLE-REPLICA-IDENTITY 2: https://www.postgresql.org/docs/current/logical-replication-publication.html

Re: set_cheapest without checking pathlist

2024-02-01 Thread James Coleman
On Thu, Feb 1, 2024 at 1:36 AM Richard Guo wrote: > > > On Thu, Feb 1, 2024 at 11:37 AM David Rowley wrote: >> >> On Thu, 1 Feb 2024 at 16:29, Richard Guo wrote: >> > On Thu, Feb 1, 2024 at 10:04 AM James Coleman wrote: >> >> I don't see any

Re: Parallelize correlated subqueries that execute within each worker

2024-01-31 Thread James Coleman
On Wed, Jan 31, 2024 at 3:18 PM Robert Haas wrote: > > On Tue, Jan 30, 2024 at 9:56 PM James Coleman wrote: > > I don't follow the "Idle since July" since it just hasn't received > > review since then, so there's been nothing to reply to. > > It

set_cheapest without checking pathlist

2024-01-31 Thread James Coleman
e any reason why this couldn't happen separately. That being said, on master I don't have a case showing this is necessary. Thanks, James Coleman 1: https://www.postgresql.org/message-id/flat/CAAaqYe-Aq6oNf9NPZnpPE7SgRLomXXWJA1Gz9L9ndi_Nv%3D94Yw%40mail.gmail.com#e0b1a803d0fdb971

Re: Parallelize correlated subqueries that execute within each worker

2024-01-31 Thread James Coleman
On Tue, Jan 30, 2024 at 10:34 PM Tom Lane wrote: > > James Coleman writes: > > I've finally had a chance to look at this, and I don't believe there's > > any real failure here, merely drift of how the planner works on master > > resulting in this query

Re: Parallelize correlated subqueries that execute within each worker

2024-01-30 Thread James Coleman
so there's been nothing to reply to. That being said, Vignesh's note in January about a now-failing test is relevant activity, and I've just today responded to that, so I'm changing the status back from Waiting on Author to Needs Review. Regards, James Coleman

Re: Parallelize correlated subqueries that execute within each worker

2024-01-30 Thread James Coleman
ross worker boundaries. However looking at the diff in the patch at that point (v10) that particular test query formed a different plan shape (there were two gather nodes being created, and params crossing between them). But in the current revision of master with the current patch applied that's

Re: Opportunistically pruning page before update

2024-01-29 Thread James Coleman
On Fri, Jan 26, 2024 at 8:33 PM James Coleman wrote: > > On Tue, Jan 23, 2024 at 2:46 AM Dilip Kumar wrote: > > > > On Tue, Jan 23, 2024 at 7:18 AM James Coleman wrote: > > > > > > On Mon, Jan 22, 2024 at 8:21 PM James Coleman wrote: > > > > &

Re: Opportunistically pruning page before update

2024-01-26 Thread James Coleman
On Tue, Jan 23, 2024 at 2:46 AM Dilip Kumar wrote: > > On Tue, Jan 23, 2024 at 7:18 AM James Coleman wrote: > > > > On Mon, Jan 22, 2024 at 8:21 PM James Coleman wrote: > > > > > > See rebased patch attached. > > > > I just realized I left a c

Re: Opportunistically pruning page before update

2024-01-22 Thread James Coleman
On Mon, Jan 22, 2024 at 8:21 PM James Coleman wrote: > > See rebased patch attached. I just realized I left a change in during the rebase that wasn't necessary. v4 attached. Regards, James Coleman v4-0002-Opportunistically-prune-to-avoid-building-a-new-p.patch Description: Binar

Re: Opportunistically pruning page before update

2024-01-22 Thread James Coleman
f necessary. > > == > [1] https://commitfest.postgresql.org/46/4384// > [2] > https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/46/4384 See rebased patch attached. Thanks, James Coleman v3-0001-Allow-getting-lock-before-calling-heap_page_prune.patch Descripti

Re: Add last_commit_lsn to pg_stat_database

2024-01-22 Thread James Coleman
f necessary. > > == > [1] > https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/46/4355 > > Kind Regards, > Peter Smith. Updated patch attached, Thanks, James Coleman v3-0001-Add-last-commit-s-LSN-to-pg_stat_database.patch Description: Binary data

Re: PG12 change to DO UPDATE SET column references

2024-01-20 Thread James Coleman
On Sat, Jan 20, 2024 at 5:57 PM Tom Lane wrote: > > James Coleman writes: > > On Sat, Jan 20, 2024 at 12:59 PM Tom Lane wrote: > >> A HINT if the bogus column name (1) matches the relation name and > >> (2) is field-qualified seems plausible to me. Then it

Re: PG12 change to DO UPDATE SET column references

2024-01-20 Thread James Coleman
On Sat, Jan 20, 2024 at 12:59 PM Tom Lane wrote: > > James Coleman writes: > > I do wonder if it's plausible (and sufficiently easy) to improve the > > error message here. "column 'foo' of relation 'foo'" makes one thing > > that you&#

Re: PG12 change to DO UPDATE SET column references

2024-01-20 Thread James Coleman
On Sat, Jan 20, 2024 at 11:12 AM Tom Lane wrote: > > James Coleman writes: > > Suppose I have this table: > > create table foo (id int primary key); > > > On PG11 this works: > > postgres=# insert into foo (id) values (1) on conflict (id) do update > > set

Re: PG12 change to DO UPDATE SET column references

2024-01-20 Thread James Coleman
On Fri, Jan 19, 2024 at 1:53 PM David G. Johnston wrote: > > On Fri, Jan 19, 2024 at 10:01 AM James Coleman wrote: >> >> Making this more confusing is the fact that if I want to do something >> like "SET bar = foo.bar + 1" the table qualification cannot be prese

PG12 change to DO UPDATE SET column references

2024-01-19 Thread James Coleman
cation on the read column the error is more understandable: ERROR: column reference "bar" is ambiguous It seems to me that it'd be desirable to either allow the unnecessary qualification or give an error that's more easily understood. Regards, James Coleman

Re: Add last_commit_lsn to pg_stat_database

2024-01-17 Thread James Coleman
On Sun, Jan 14, 2024 at 6:01 AM vignesh C wrote: > > On Sat, 10 Jun 2023 at 07:57, James Coleman wrote: > > > > I've previously noted in "Add last commit LSN to > > pg_last_committed_xact()" [1] that it's not possible to monitor how > > many

Re: Add last_commit_lsn to pg_stat_database

2024-01-17 Thread James Coleman
mit_lsn() is marked as STABLE which is > probably correct. But I would appreciate a second opinion on this. Sounds good. > - Wouldn't it be appropriate to add a test or two? Added. > - `if (!XLogRecPtrIsInvalid(commit_lsn))` - I suggest adding > XLogRecPtrIsValid() macro for better read

Re: Teach predtest about IS [NOT] proofs

2024-01-17 Thread James Coleman
On Fri, Dec 22, 2023 at 2:48 PM Tom Lane wrote: > > James Coleman writes: > > I've not yet applied all of your feedback, but I wanted to get an > > initial read on your thoughts on how using switch statements ends up > > looking. Attached is a single (pure refa

Re: Teach predtest about IS [NOT] proofs

2023-12-22 Thread James Coleman
On Thu, Dec 14, 2023 at 4:38 PM Tom Lane wrote: > > James Coleman writes: > > On Wed, Dec 13, 2023 at 1:36 PM Tom Lane wrote: > >> I don't have an objection in principle to adding more smarts to > >> predtest.c. However, we should be wary of slowing down c

Re: Teach predtest about IS [NOT] proofs

2023-12-13 Thread James Coleman
Thanks for taking a look! On Wed, Dec 13, 2023 at 1:36 PM Tom Lane wrote: > > James Coleman writes: > > Attached is a patch that solves that issue. It also teaches predtest about > > quite a few more cases involving BooleanTest expressions (e.g., how they > > relate

Teach predtest about IS [NOT] proofs

2023-12-11 Thread James Coleman
o run the "x, y" case as well as the "y, x" case with a single call so as to eliminate a lot of repetition in clause/expression test cases. If reviewers agree that's desirable, then I could do that as a precursor. Regards, James Coleman v1-0001-Teach-predtest-about-IS-NOT-bool-proofs.patch Description: Binary data

Re: RFC: Logging plan of the running query

2023-10-18 Thread James Coleman
ese states may leak depending upon when this > >>> function gets called. > >>> 3. Building features on top as James envisions will be easier. In my view the fact that auto_explain is itself not part of core is a mistake, and I know there are more prominent hackers than myself who hold that view. While that decision as regards auto_explain has long since been made (and there would be work to undo it), I don't believe that we should repeat that choice here. I'm -10 on moving this into auto_explain. However perhaps there is still an opportunity for moving some of the auto_explain code into core so as to enable deduplicating the code. Regards, James Coleman

Re: RFC: Logging plan of the running query

2023-10-06 Thread James Coleman
On Fri, Oct 6, 2023 at 8:58 AM torikoshia wrote: > > On 2023-10-04 03:00, James Coleman wrote: > > On Thu, Sep 7, 2023 at 2:09 AM torikoshia > > wrote: > >> > >> On 2023-09-06 11:17, James Coleman wrote: > >> > >> >> > I

Re: Opportunistically pruning page before update

2023-10-06 Thread James Coleman
Hi, Thanks for taking a look! On Fri, Oct 6, 2023 at 1:18 AM Dilip Kumar wrote: > > On Thu, Oct 5, 2023 at 2:35 AM James Coleman wrote: > > > > I talked to Andres and Peter again today, and out of that conversation > > I have some observations and ideas for future impr

Re: [DOCS] HOT - correct claim about indexes not referencing old line pointers

2023-10-04 Thread James Coleman
On Wed, Oct 4, 2023 at 9:42 AM Robert Haas wrote: > > On Wed, Oct 4, 2023 at 9:36 AM James Coleman wrote: > > Are you thinking we should simply elide the fact that there is pruning > > that happens outside of HOT? Or add that information onto the HOT > > page, even though

Re: Opportunistically pruning page before update

2023-10-04 Thread James Coleman
On Tue, Sep 26, 2023 at 8:30 AM James Coleman wrote: > > On Tue, Sep 5, 2023 at 1:40 PM Melanie Plageman > wrote: > > > > On Wed, Jun 21, 2023 at 8:51 AM James Coleman wrote: > > > While at PGCon I was chatting with Andres (and I think Peter G. and a > > >

Re: [DOCS] HOT - correct claim about indexes not referencing old line pointers

2023-10-04 Thread James Coleman
On Wed, Oct 4, 2023 at 9:18 AM Robert Haas wrote: > > On Tue, Oct 3, 2023 at 3:35 PM James Coleman wrote: > > I like your changes. Reading through this several times, and noting > > Peter's comments about pruning being more than just HOT, I'm thinking > > that r

Re: [DOCS] HOT - correct claim about indexes not referencing old line pointers

2023-10-03 Thread James Coleman
On Mon, Oct 2, 2023 at 2:55 PM Robert Haas wrote: > > On Sat, Sep 30, 2023 at 1:05 AM Peter Geoghegan wrote: > > > This is why I discovered it: it says "indexes do not reference their > > > page item identifiers", which is manifestly not true when talking > > > about the root item, and in fact wo

Re: RFC: Logging plan of the running query

2023-10-03 Thread James Coleman
On Thu, Sep 7, 2023 at 2:09 AM torikoshia wrote: > > On 2023-09-06 11:17, James Coleman wrote: > > >> > I've never been able to reproduce it (haven't tested the new version, > >> > but v28 at least) on my M1 Mac; where I've reproduced it is

Re: [DOCS] HOT - correct claim about indexes not referencing old line pointers

2023-09-29 Thread James Coleman
On Fri, Sep 29, 2023 at 4:06 PM Peter Geoghegan wrote: > > On Fri, Sep 29, 2023 at 11:45 AM James Coleman > wrote:my reading the issue is that "old versions" doesn't say > > anything about "old HOT versions; it seems to be describing what > > happens ge

Re: [DOCS] HOT - correct claim about indexes not referencing old line pointers

2023-09-29 Thread James Coleman
ten -- which would include the first time a heap-only tuple is written. And when it's the first heap-only tuple the "old version" would be the original version, which would not be a heap-only tuple. I can work up a tweaked version of the patch that shows there are two paths here (original tuple is being updated versus an intermediate heap-only tuple is being updated); would you be willing to consider that? Thanks, James Coleman

Re: Fix incorrect comment reference

2023-09-29 Thread James Coleman
On Fri, Sep 29, 2023 at 2:26 PM Bruce Momjian wrote: > > On Mon, Jan 23, 2023 at 06:42:45PM -0500, James Coleman wrote: > > On Mon, Jan 23, 2023 at 4:07 PM James Coleman wrote: > > > > > > On Mon, Jan 23, 2023 at 3:41 PM Robert Haas wrote: > > > > >

[DOCS] HOT - correct claim about indexes not referencing old line pointers

2023-09-29 Thread James Coleman
ng this correctly, attached is a patch correcting the description. Thanks, James Coleman v1-0001-Correct-HOT-docs-to-account-for-LP_REDIRECT.patch Description: Binary data

Re: Opportunistically pruning page before update

2023-09-26 Thread James Coleman
On Tue, Sep 5, 2023 at 1:40 PM Melanie Plageman wrote: > > On Wed, Jun 21, 2023 at 8:51 AM James Coleman wrote: > > While at PGCon I was chatting with Andres (and I think Peter G. and a > > few others who I can't remember at the moment, apologies) and Andres

Re: RFC: Logging plan of the running query

2023-09-05 Thread James Coleman
On Tue, Sep 5, 2023 at 9:59 AM torikoshia wrote: > > On 2023-08-28 22:47, James Coleman wrote: > > On Mon, Aug 28, 2023 at 3:01 AM torikoshia > > wrote: > >> > >> On 2023-08-26 21:03, James Coleman wrote: > >> > On Fri, Aug 25, 2023 at 7:43 AM Jame

Re: RFC: Logging plan of the running query

2023-08-28 Thread James Coleman
On Mon, Aug 28, 2023 at 3:01 AM torikoshia wrote: > > On 2023-08-26 21:03, James Coleman wrote: > > On Fri, Aug 25, 2023 at 7:43 AM James Coleman wrote: > >> > >> On Thu, Aug 17, 2023 at 10:02 AM torikoshia > >> wrote: > >> > > >> >

Re: RFC: Logging plan of the running query

2023-08-26 Thread James Coleman
On Fri, Aug 25, 2023 at 7:43 AM James Coleman wrote: > > On Thu, Aug 17, 2023 at 10:02 AM torikoshia > wrote: > > > > On 2023-06-16 01:34, James Coleman wrote: > > > Attached is v28 > > > which sets ProcessLogQueryPlanInterruptActive to false in errfinis

Re: RFC: Logging plan of the running query

2023-08-25 Thread James Coleman
On Thu, Aug 17, 2023 at 10:02 AM torikoshia wrote: > > On 2023-06-16 01:34, James Coleman wrote: > > Attached is v28 > > which sets ProcessLogQueryPlanInterruptActive to false in errfinish > > when necessary. Once built with those two patches I'm simply running > &

Re: pgindent (probably my missing something obvious)

2023-07-03 Thread James Coleman
On Mon, Jul 3, 2023 at 9:20 PM Tom Lane wrote: > > James Coleman writes: > > This is the first time I've run pgindent on my current machine, and it > > doesn't seem to be making any modifications to source files. For > > example this command: > > >

Re: Parallelize correlated subqueries that execute within each worker

2023-07-03 Thread James Coleman
On Sun, Jun 11, 2023 at 10:23 PM James Coleman wrote: > > ... > > And while trying the v9 patch I came across a crash with the query > > below. > > > > set min_parallel_table_scan_size to 0; > > set parallel_setup_cost to 0; > > set parallel_tuple_cost to

pgindent (probably my missing something obvious)

2023-07-03 Thread James Coleman
aving some very long function calls. I've downloaded the latest typedefs list, but I haven't added any types anyway. What obvious thing am I missing? Thanks, James Coleman

Re: Analyze on table creation?

2023-06-27 Thread James Coleman
On Mon, Jun 26, 2023 at 4:16 PM James Coleman wrote: > > On Mon, Jun 26, 2023 at 4:00 PM Andres Freund wrote: > > > > Hi, > > > > On 2023-06-26 13:40:49 -0400, James Coleman wrote: > > > Have we ever discussed running an analyze immediately after creati

Re: Analyze on table creation?

2023-06-26 Thread James Coleman
On Mon, Jun 26, 2023 at 4:00 PM Andres Freund wrote: > > Hi, > > On 2023-06-26 13:40:49 -0400, James Coleman wrote: > > Have we ever discussed running an analyze immediately after creating a > > table? > > That doesn't make a whole lot of sense to me - we coul

Re: Analyze on table creation?

2023-06-26 Thread James Coleman
cc'ing Tom because I'm curious if he's willing to provide some greater context on the commit in question. On Mon, Jun 26, 2023 at 2:16 PM Pavel Stehule wrote: > > > > po 26. 6. 2023 v 19:48 odesílatel James Coleman napsal: >> >> On Mon, Jun 26, 2

Re: Analyze on table creation?

2023-06-26 Thread James Coleman
On Mon, Jun 26, 2023 at 1:45 PM Pavel Stehule wrote: > > > > po 26. 6. 2023 v 19:43 odesílatel Pavel Stehule > napsal: >> >> Hi >> >> po 26. 6. 2023 v 19:41 odesílatel James Coleman napsal: >>> >>> Hello, >>> >>> Have w

Analyze on table creation?

2023-06-26 Thread James Coleman
so I assume people have considered it before. If so, I'd like to understand why the conclusion was not to do it, or, alternatively if it's a lack of tuits. Regards, James Coleman

Re: Stampede of the JIT compilers

2023-06-25 Thread James Coleman
does, but I don't > have a Google Cloud SQL or RDS instance running right to verify their > settings. I do seem to remember that they did as well though, at least a > while back. > > > Michael I believe it's off by default in Aurora Postgres also. Regards, James Coleman

Re: Stampede of the JIT compilers

2023-06-24 Thread James Coleman
On Sat, Jun 24, 2023 at 8:14 PM David Rowley wrote: > > On Sun, 25 Jun 2023 at 05:54, Tom Lane wrote: > > > > James Coleman writes: > > > On Sat, Jun 24, 2023 at 7:40 AM Tomas Vondra > > > wrote: > > >> On 6/24/23 02:33, David Rowley wrote: >

Re: Stampede of the JIT compilers

2023-06-24 Thread James Coleman
On Sat, Jun 24, 2023 at 1:54 PM Tom Lane wrote: > > James Coleman writes: > > In that context capping the number of backends compiling, particularly > > where plans (and JIT?) might be cached, could well save us (depending > > on workload). > > TBH I do not find t

Re: Stampede of the JIT compilers

2023-06-24 Thread James Coleman
On Sat, Jun 24, 2023 at 7:40 AM Tomas Vondra wrote: > > > > On 6/24/23 02:33, David Rowley wrote: > > On Sat, 24 Jun 2023 at 02:28, James Coleman wrote: > >> There are a couple of issues here. I'm sure it's been discussed > >> before, and it'

Stampede of the JIT compilers

2023-06-23 Thread James Coleman
ered implemented a GUC/feature like "max_concurrent_jit_compilations" to cap the number of backends that may be compiling a query at any given point so that we avoid an optimization from running amok and consuming all of a servers resources? Regards, James Coleman

Re: Memory leak in incremental sort re-scan

2023-06-21 Thread James Coleman
sorted_keys). That avoids unnecessary recreation of the sort states, but it also fixes the problem Tom noted as well: the call to preparePresortedCols() is already guarded by a test on fullsort_state being NULL, so with this change we also won't unnecessarily redo that work. Regards, James Coleman v2-0001-Fix-memory-leak-in-incremental-sort-rescan.patch Description: Binary data

Re: Use of additional index columns in rows filtering

2023-06-21 Thread James Coleman
On Wed, Jun 21, 2023 at 11:28 AM Tomas Vondra wrote: > > > > On 6/21/23 14:45, James Coleman wrote: > > Hello, > > > > I've cc'd Jeff Davis on this due to a conversation we had at PGCon > > about applying filters on index tuples during index scans.

Opportunistically pruning page before update

2023-06-21 Thread James Coleman
t had time to devote properly to that, so I'm wondering if there's anyone who might be interested in collaborating on that part. Other TODOs: - Audit other callers of RelationSetTargetBlock() to ensure they don't hold pointers into the page. Regards, James Coleman v1-000

Re: Use of additional index columns in rows filtering

2023-06-21 Thread James Coleman
Rows Removed by Index Recheck: 197780 > Filter: (b = 4) > Buffers: shared hit=544 > Planning Time: 0.105 ms > Execution Time: 13.690 ms > (10 rows) > > ... I did also confirm that this properly identifies cases Jeff had mentioned to me like "Index Filter: (((a * 2) > 50) AND ((b % 10) = 4))". I noticed also you still had questions/TODOs about handling index scans for join clauses. Regards, James Coleman 1: https://www.postgresql.org/message-id/20230609000600.syqy447e6metnvyj%40awork3.anarazel.de

Re: path->param_info only set for lateral?

2023-06-20 Thread James Coleman
On Sun, Jun 18, 2023 at 10:57 PM Tom Lane wrote: > > James Coleman writes: > > Over in "Parallelize correlated subqueries that execute within each > > worker" [1} Richard Guo found a bug in the current version of my patch > > in that thread. While debugging

path->param_info only set for lateral?

2023-06-18 Thread James Coleman
ateral reference) we're not going to get any ParamPathInfo added to the path or the rel. Is there a reason why we don't track the required relids providing the PARAM_EXEC params in this case? Thanks, James Coleman 1: https://www.postgresql.org/message-id/CAMbWs4_evjcMzN8Gw78bHfhfo2FKJThqhEjRJRmoMZx%3DNXcJ7w%40mail.gmail.com

Re: RFC: Logging plan of the running query

2023-06-15 Thread James Coleman
On Thu, Jun 15, 2023 at 9:00 AM torikoshia wrote: > > On 2023-06-15 01:48, James Coleman wrote: > > On Tue, Jun 13, 2023 at 11:53 AM James Coleman > > wrote: > >> > >> ... > >> I'm going to re-run tests with my patch version + resetting the flag

Re: RFC: Logging plan of the running query

2023-06-14 Thread James Coleman
On Tue, Jun 13, 2023 at 11:53 AM James Coleman wrote: > > ... > I'm going to re-run tests with my patch version + resetting the flag > on SIGINT (and any other error condition) to be certain that the issue > you uncovered (where backends get stuck after a SIGINT not respondin

Re: RFC: Logging plan of the running query

2023-06-13 Thread James Coleman
On Tue, Jun 13, 2023 at 11:22 AM torikoshia wrote: > > On 2023-06-13 00:52, James Coleman wrote: > >> > >> > I've attached v27. The important change here in 0001 is that it > >> > guarantees the interrupt handler is re-entrant, since that was a bug >

Re: RFC: Logging plan of the running query

2023-06-12 Thread James Coleman
On Sun, Jun 11, 2023 at 11:07 PM torikoshia wrote: > > On 2023-06-06 03:26, James Coleman wrote: > > On Mon, Jun 5, 2023 at 4:30 AM torikoshia > > wrote: > >> > >> On 2023-06-03 02:51, James Coleman wrote: > >> > Hello, > >> > >

Re: Parallelize correlated subqueries that execute within each worker

2023-06-11 Thread James Coleman
On Tue, Jun 6, 2023 at 4:36 AM Richard Guo wrote: > > > On Mon, Jan 23, 2023 at 10:00 PM James Coleman wrote: >> >> Which this patch we do in fact now see (as expected) rels with >> non-empty lateral_relids showing up in generate_[useful_]gather_paths. >> And t

Add last_commit_lsn to pg_stat_database

2023-06-09 Thread James Coleman
) fits more naturally into the stats system. I'd originally thought of exposing it in pg_stat_wal, but that's per-cluster rather than per-database (indeed, this is a flaw I hadn't considered in the original patch), so I think pg_stat_database is the correct location. I've

Re: RFC: Logging plan of the running query

2023-06-05 Thread James Coleman
On Mon, Jun 5, 2023 at 4:30 AM torikoshia wrote: > > On 2023-06-03 02:51, James Coleman wrote: > > Hello, > > > > Thanks for working on this patch! Sure thing! I'm *very interested* in seeing this available, and I think it paves the way for some additional features l

Re: RFC: Logging plan of the running query

2023-06-02 Thread James Coleman
ning, and I think it would be worth adding that. Are you interested in re-opening this patch? I'd be happy to provide further review and help to try to push this along. I've rebased the patch and attached as v26. Thanks, James Coleman v26-0001-Add-function-to-log-the-plan-of-the-query.patch Description: Binary data

Re: An inefficient query caused by unnecessary PlaceHolderVar

2023-06-01 Thread James Coleman
On Wed, May 31, 2023 at 10:30 PM Richard Guo wrote: > > > On Wed, May 31, 2023 at 1:27 AM James Coleman wrote: >> >> This looks good to me. > > > Thanks for the review! Sure thing! >> >> A few small tweaks suggested to comment wording: >> &g

Re: An inefficient query caused by unnecessary PlaceHolderVar

2023-05-30 Thread James Coleman
d rel is under the same lowest nulling outer + * join. I think this is clearer: "references something outside the subquery being pulled up and is not under the same lowest outer join." One other thing: it would be helpful to have the test query output be stable between HEAD and this patch; perhaps add: order by 1, 2, 3, 4, 5, 6, 7 to ensure stability? Thanks, James Coleman

Re: pg_rewind: warn when checkpoint hasn't happened after promotion

2023-02-28 Thread James Coleman
On Mon, Feb 27, 2023 at 2:33 AM Heikki Linnakangas wrote: > > On 16/11/2022 07:17, kuroda.keis...@nttcom.co.jp wrote: > >> The issue here is pg_rewind looks into control file to determine the > >> soruce timeline, because the control file is not updated until the > >> first checkpoint ends after p

Re: Parallelize correlated subqueries that execute within each worker

2023-02-08 Thread James Coleman
On Mon, Feb 6, 2023 at 11:39 AM Antonin Houska wrote: > > James Coleman wrote: > > Which this patch we do in fact now see (as expected) rels with > > non-empty lateral_relids showing up in generate_[useful_]gather_paths. > > And the partial paths can now have non-empty

Re: Fix incorrect comment reference

2023-01-23 Thread James Coleman
On Mon, Jan 23, 2023 at 4:07 PM James Coleman wrote: > > On Mon, Jan 23, 2023 at 3:41 PM Robert Haas wrote: > > > > On Mon, Jan 23, 2023 at 3:19 PM James Coleman wrote: > > > On Mon, Jan 23, 2023 at 1:26 PM Robert Haas wrote: > > > > On Mon, Jan 2

  1   2   3   4   5   6   >