Re: Partial aggregates pushdown

2024-07-08 Thread Jelte Fennema-Nio
On Mon, 8 Jul 2024 at 14:12, fujii.y...@df.mitsubishielectric.co.jp wrote: > The best thing about Approach2 is that it lets us send state values using the > existing data type system. > I'm worried that if we choose Approach2, we might face some limits because we > have to create new types. >

Re: Partial aggregates pushdown

2024-07-08 Thread Jelte Fennema-Nio
On Sun, 7 Jul 2024 at 23:52, fujii.y...@df.mitsubishielectric.co.jp wrote: > My plan for advancing the patch involves the following steps: > Step1. Decide the approach on transmitting state value. > Step2. Implement code (including comments) and tests to support a subset of > aggregate

Re: Partial aggregates pushdown

2024-07-08 Thread Jelte Fennema-Nio
On Sun, 30 Jun 2024 at 23:42, fujii.y...@df.mitsubishielectric.co.jp wrote: > Instead, I can make PARTIAL_AGGREGATE an unreserved word by placing it after > the FILTER clause, like avg(c1) FILTER (WHERE c2 > 0) PARTIAL_AGGREGATE, and > by marking it as an ASLABEL word like FILTER. > I attached

Re: Partial aggregates pushdown

2024-07-08 Thread Jelte Fennema-Nio
ch may be seen as cumbersome by some > developers. > Option2 restricts developers to using only text representation for > transmitting state values, which I consider limiting. In my opinion this is your strongest argument for Approach1. But you didn't answer my previous two questions yet: On

Re: Make query cancellation keys longer

2024-07-04 Thread Jelte Fennema-Nio
On Thu, 4 Jul 2024 at 14:43, Tomas Vondra wrote: > I don't have any immediate feedback regarding this patch, but I'm > wondering about one thing related to cancellations - we talk cancelling > a query, but we really target a PID (or a particular backend, no matter > how we identify it). > > I

Re: Make query cancellation keys longer

2024-07-04 Thread Jelte Fennema-Nio
On Thu, 4 Jul 2024 at 12:32, Heikki Linnakangas wrote: > We currently don't do any locking on the ProcSignal array. For query > cancellations, that's good because a query cancel packet is processed > without having a PGPROC entry, so we cannot take LWLocks. We could use > spinlocks though. In

Re: Make query cancellation keys longer

2024-07-04 Thread Jelte Fennema-Nio
On Thu, 4 Jul 2024 at 12:35, Heikki Linnakangas wrote: > > On 04/07/2024 13:32, Heikki Linnakangas wrote: > > Here's a new version of the first patch. > > Sorry, forgot attachment. It seems you undid the following earlier change. Was that on purpose? If not, did you undo any other earlier

Re: Fix a comment on PQcancelErrorMessage

2024-07-04 Thread Jelte Fennema-Nio
On Thu, 4 Jul 2024 at 06:46, Yugo NAGATA wrote: > Attached is a small patch to fix a comment on PQcancelErrorMessage. Oops, copy paste mistake on my part I guess. New comment LGTM

Re: Add a GUC check hook to ensure summarize_wal cannot be enabled when wal_level is minimal

2024-07-03 Thread Jelte Fennema-Nio
On Wed, 3 Jul 2024 at 16:46, Jelte Fennema-Nio wrote: > > Ugh copy paste mistake, this is what I meant > > On Wed, 3 Jul 2024 at 16:45, Jelte Fennema-Nio wrote: > > This hits the already existing check: > > summarize_wal = 'true' > > wal_level = 'minimal' &

Re: Add a GUC check hook to ensure summarize_wal cannot be enabled when wal_level is minimal

2024-07-03 Thread Jelte Fennema-Nio
Ugh copy paste mistake, this is what I meant On Wed, 3 Jul 2024 at 16:45, Jelte Fennema-Nio wrote: > This hits the already existing check: > summarize_wal = 'true' > wal_level = 'minimal' > > This hits the new check: > summarize_wal = 'true' > wal_level = 'minimal' > &

Re: Add a GUC check hook to ensure summarize_wal cannot be enabled when wal_level is minimal

2024-07-03 Thread Jelte Fennema-Nio
On Wed, 3 Jul 2024 at 16:30, Nathan Bossart wrote: > IME these sorts of GUC hooks that depend on the value of other GUCs tend to > be quite fragile. This one might be okay because wal_level defaults to > 'replica' and because there is an additional check in postmaster.c, but > that at least

Re: [PATCH] Handle SK_SEARCHNULL and SK_SEARCHNOTNULL in HeapKeyTest

2024-07-02 Thread Jelte Fennema-Nio
On Tue, 2 Jul 2024 at 10:15, Aleksander Alekseev wrote: > The referred patch was rejected at first because it didn't modify > nodeSeqScan.c to make use of the change within the core. I guess we interpret Heikis email differently. I read it as: "If this improves performance, then let's also start

Re: [PATCH] Handle SK_SEARCHNULL and SK_SEARCHNOTNULL in HeapKeyTest

2024-07-02 Thread Jelte Fennema-Nio
On Mon, 1 Jul 2024 at 15:48, Aleksander Alekseev wrote: > As I recall, previously it was argued that changes like this should > have some use within the core [1]. I don't see that argument anywhere in the thread honestly. I did see heiki asking why it would be useful for extensions, but that was

Re: doc: modify the comment in function libpqrcv_check_conninfo()

2024-07-01 Thread Jelte Fennema-Nio
On Thu, 27 Jun 2024 at 12:27, ikedarintarof wrote: > Thanks for your suggestion. I used ChatGPT to choose the wording, but > it's still difficult for me. Looks good to me now (but obviously biased since you took my wording). Adding Robert, since he authored the commit that introduced this

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-06-30 Thread Jelte Fennema-Nio
On Sun, 30 Jun 2024 at 21:00, Noah Misch wrote: > Shouldn't that be s/conn->status/cancelConn->status/? Ugh yes, I think this was a copy paste error. See attached patch 0003 to fix this (rest of the patches are untouched from previous revision).

Re: Converting README documentation to Markdown

2024-06-29 Thread Jelte Fennema-Nio
On Fri, 28 Jun 2024 at 20:40, Peter Eisentraut wrote: > I have my "less" set up so that "less somefile.md" automatically renders > the markdown. That's been pretty useful. But if that now keeps making > a mess out of PostgreSQL's README files, then I'm going to have to keep > fixing things, and

Re: cfbot update: Using GitHub for patch review

2024-06-29 Thread Jelte Fennema-Nio
On Sat, 29 Jun 2024 at 01:13, Thomas Munro wrote: > > On Sat, Jun 29, 2024 at 1:10 AM Ashutosh Bapat > wrote: > > I need to sign in to github to add my review comments. So those who do not > > have a github account can not use it for review. But I don't think that can > > be fixed. We need a

Re: Converting README documentation to Markdown

2024-06-28 Thread Jelte Fennema-Nio
On Fri, 28 Jun 2024 at 09:38, Peter Eisentraut wrote: > Getting that right in Markdown can be quite tricky. I agree that in some cases it's tricky. But what's the worst case that can happen when you get it wrong? It renders weird on github.com. Luckily there's a "code" button to go to the plain

Re: Improve EXPLAIN output for multicolumn B-Tree Index

2024-06-28 Thread Jelte Fennema-Nio
On Fri, 28 Jun 2024 at 00:41, Peter Geoghegan wrote: > Typically, no, it won't be. But there's really no telling for sure. > The access patterns for a composite index on '(a, b)' with a qual > "WHERE b = 5" are identical to a qual explicitly written "WHERE a = > any() AND b = 5". Hmm, that's

Re: Improve EXPLAIN output for multicolumn B-Tree Index

2024-06-27 Thread Jelte Fennema-Nio
On Thu, 27 Jun 2024 at 22:02, Peter Geoghegan wrote: > It's also possible that we should just do something simple, like your > patch, even though technically it won't really be accurate in cases > where skip scan is used to good effect. Maybe showing the "default > working assumption" about how

Re: ClientRead on ROLLABACK

2024-06-27 Thread Jelte Fennema-Nio
On Thu, 27 Jun 2024 at 17:21, Tom Lane wrote: > (We used to show "" in the query column in this state, but that > was deemed less helpful than the current behavior.) I think this is a super common confusion among users. Maybe we should consider making it clearer that no query is currently being

Re: Support a wildcard in backtrace_functions

2024-06-27 Thread Jelte Fennema-Nio
wo GUCs instead of one to do what I want. From 85a894cc32381a4ff2a1c7aab31476b2bbf6bdf3 Mon Sep 17 00:00:00 2001 From: Jelte Fennema-Nio Date: Thu, 21 Mar 2024 13:05:35 +0100 Subject: [PATCH 1/2] Add PGErrorVerbosity to typedefs.list This one was missing, resulting in some strange alignment. --- sr

Re: doc: modify the comment in function libpqrcv_check_conninfo()

2024-06-27 Thread Jelte Fennema-Nio
On Thu, 27 Jun 2024 at 09:09, ikedarintarof wrote: > > Thank you for your comment! > > I've added the must_use_password argument. A new patch is attached. s/whther/whether nit: "it will do nothing" sounds a bit strange to me (but I'm not native english). Something like this reads more natural

Re: libpq: Fix lots of discrepancies in PQtrace

2024-06-27 Thread Jelte Fennema-Nio
On Thu, 27 Jun 2024 at 07:39, Michael Paquier wrote: > Looking at the whole, the rest of the patch set qualifies as a new > feature, even if they're aimed at closing existing gaps. Alright, seems reasonable. To be clear, I don't care at all about this being backported personally. >

Re: libpq: Fix lots of discrepancies in PQtrace

2024-06-26 Thread Jelte Fennema-Nio
On Wed, 26 Jun 2024 at 18:36, Alvaro Herrera wrote: > Thanks, Nathan. I'm holding myself responsible for the rest ... will > handle soon after the branch. Sounds great. Out of curiosity, what is the backpatching policy for something like this? Honestly most of these patches could be considered

Re: doc: modify the comment in function libpqrcv_check_conninfo()

2024-06-26 Thread Jelte Fennema-Nio
On Wed, 26 Jun 2024 at 14:53, ikedarintarof wrote: > The function 'libpqrcv_check_conninfo()' returns 'void', but the comment > above says that the function returns true or false. > I've attached a patch to modify the comment. Agreed that the current comment is wrong, but the new comment should

Re: RFC: Additional Directory for Extensions

2024-06-26 Thread Jelte Fennema-Nio
On Tue, 25 Jun 2024 at 19:33, David E. Wheeler wrote: > > On Jun 24, 2024, at 5:32 PM, Jelte Fennema-Nio wrote: > > > Still, for the sake of completeness it might make sense to support > > this whole list in extension_destdir. (assuming it's easy to do) > > It shoul

Re: RFC: Additional Directory for Extensions

2024-06-26 Thread Jelte Fennema-Nio
On Wed, 26 Jun 2024 at 00:32, David E. Wheeler wrote: > In other news, here’s an updated patch that expands the documentation to > record that the destination directory is a prefix, and full paths should be > used under it. Also take the opportunity to document the PGXS DESTDIR > variable as

Re: RFC: Additional Directory for Extensions

2024-06-25 Thread Jelte Fennema-Nio
On Tue, 25 Jun 2024 at 12:12, Alvaro Herrera wrote: > They can mutilate the system catalogs: yes, they can TRUNCATE pg_type. > So what? They've just destroyed their own ability to do anything else. > The real issue here is that they can edit pg_proc to cause SQL function > calls to call

Re: Partial aggregates pushdown

2024-06-25 Thread Jelte Fennema-Nio
On Tue, 25 Jun 2024 at 08:33, fujii.y...@df.mitsubishielectric.co.jp wrote: > Actually, I have other tasks about "PARTIAL_AGGREAGATE" keyword > to respond Requirement1 and Requirement2 in the following mail. >

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-06-25 Thread Jelte Fennema-Nio
On Tue, 25 Jun 2024 at 07:00, Alexander Lakhin wrote: > I'd just like to add that that one original query assumes several "remote" > queries (see the attached excerpt from postmaster.log with verbose logging > enabled). Nice catch! All those EXPLAIN queries are definitely not intentional, and

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Jelte Fennema-Nio
On Mon, 24 Jun 2024 at 22:42, David E. Wheeler wrote: > >> BINDIR > >> DOCDIR > >> HTMLDIR > >> PKGINCLUDEDIR > >> LOCALEDIR > >> MANDIR > >> > >> I can imagine an extension wanting or needing to use any and all of these. > > > > Are these really all relevant to backend code? > > Oh I think so.

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Jelte Fennema-Nio
On Mon, 24 Jun 2024 at 18:11, Nathan Bossart wrote: > At first glance, the general idea seems reasonable to me. I'm wondering > whether there is a requirement for this directory to be prepended or if it > could be appended to the end. That way, the existing ones would take > priority, which

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Jelte Fennema-Nio
On Thu, 11 Apr 2024 at 19:52, David E. Wheeler wrote: > I realize this probably isn’t going to happen for 17, given the freeze, but I > would very much welcome feedback and pointers to address concerns about > providing a second directory for extensions and DSOs. Quite a few people have >

Re: Partial aggregates pushdown

2024-06-24 Thread Jelte Fennema-Nio
On Mon, 24 Jun 2024 at 15:03, fujii.y...@df.mitsubishielectric.co.jp wrote: > I see. I maybe got your proposal. > Refer to your proposal, for avg(int8), > I create a new native type like state_int8_avg > with the new typsend/typreceive functions > and use them to transmit the state value, right?

Re: Improve EXPLAIN output for multicolumn B-Tree Index

2024-06-24 Thread Jelte Fennema-Nio
On Mon, 24 Jun 2024 at 13:02, Matthias van de Meent wrote: > It does not really behave similar: index scan keys (such as the > id3=101 scankey) don't require visibility checks in the btree code, > while the Filter condition _does_ require a visibility check, and > delegates the check to the table

Re: Improve EXPLAIN output for multicolumn B-Tree Index

2024-06-24 Thread Jelte Fennema-Nio
+1 for the idea. On Mon, 24 Jun 2024 at 11:11, Matthias van de Meent wrote: > I think this is too easy to confuse with the pre-existing 'Filter' > condition, which you'll find on indexes with INCLUDE-d columns or > filters on non-index columns. Why not combine them? And both call them Filter?

Re: Partial aggregates pushdown

2024-06-24 Thread Jelte Fennema-Nio
On Sun, 23 Jun 2024 at 10:24, fujii.y...@df.mitsubishielectric.co.jp wrote: > I attached the POC patch(the second one) which supports avg(int8) whose > standard format is _numeric type. Okay, very interesting. So instead of defining the serialization/deserialization functions to text/binary,

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-06-23 Thread Jelte Fennema-Nio
On Sat, 22 Jun 2024 at 17:00, Alexander Lakhin wrote: > @@ -2775,6 +2775,7 @@ > SET LOCAL statement_timeout = '10ms'; > select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- > this takes very long > ERROR: canceling statement due to statement timeout > +WARNING: could

cfbot update: Using GitHub for patch review

2024-06-21 Thread Jelte Fennema-Nio
I recently got write access to the cfbot repo[1] and machine from Thomas. And I deployed a few improvements this week. The most significant one is that it is now much easier to use GitHub as part of your patch review workflow. On the cfbot website[2] there's now a "D" (diff) link next to each

Re: Small LO_BUFSIZE slows down lo_import and lo_export in libpq

2024-06-21 Thread Jelte Fennema-Nio
On Fri, 21 Jun 2024 at 10:46, Дмитрий Питаков wrote: > Why not increase the buffer size? I think changing the buffer size sounds like a reasonable idea, if that speeds stuff up. But I think it would greatly help your case if you showed the perf increase using a simple benchmark, especially if

libpq: Fix lots of discrepancies in PQtrace

2024-06-21 Thread Jelte Fennema-Nio
ort=5432' -t test.trace And then look at the top of test.trace [1]: https://www.postgresql.org/message-id/CAGECzQTTN5aGqtDaRifJXPyd_O5qHWQcOxsHJsDSVNqMugGNEA%40mail.gmail.com From 8567a09b4e1816ccdd58786f912ee055accfbc93 Mon Sep 17 00:00:00 2001 From: Jelte Fennema-Nio Date: Fri, 21 Jun 2024 01

Re: Extension security improvement: Add support for extensions with an owned schema

2024-06-20 Thread Jelte Fennema-Nio
On Wed, 19 Jun 2024 at 17:22, David G. Johnston wrote: > > On Wed, Jun 19, 2024 at 8:19 AM Jelte Fennema-Nio wrote: > >> >> Because part of it would >> only be relevant once we support upgrading from PG18. So for now the >> upgrade_code I haven't actually run

Re: Extension security improvement: Add support for extensions with an owned schema

2024-06-19 Thread Jelte Fennema-Nio
On Wed, 19 Jun 2024 at 19:55, Tom Lane wrote: > > Jelte Fennema-Nio writes: > > On Wed, 19 Jun 2024 at 17:28, Robert Haas wrote: > >> I have a feeling that this might be pretty annoying to implement, and > >> if that is true, then never mind. > > >

Re: Extension security improvement: Add support for extensions with an owned schema

2024-06-19 Thread Jelte Fennema-Nio
On Wed, 19 Jun 2024 at 17:28, Robert Haas wrote: > But I wonder if there might also be another possible approach: could > we, somehow, prevent object references in extension scripts from > resolving to anything other than the system catalogs and the contents > of that extension? This indeed does

Re: Extension security improvement: Add support for extensions with an owned schema

2024-06-19 Thread Jelte Fennema-Nio
support upgrading from PG18. So for now the upgrade_code I haven't actually run. From 37b6fa45bf877bcc15ce76d7e342199b7ca76d50 Mon Sep 17 00:00:00 2001 From: Jelte Fennema-Nio Date: Fri, 31 May 2024 02:04:31 -0700 Subject: [PATCH v2] Add support for extensions with an owned schema Writing the sql

Re: RFC: adding pytest as a supported test framework

2024-06-17 Thread Jelte Fennema-Nio
On Sun, 16 Jun 2024 at 23:04, Robert Haas wrote: > > On Sat, Jun 15, 2024 at 6:00 PM Melanie Plageman > wrote: > > Writing a new test framework in a popular language that makes it more > > likely that more people will write more tests and test infrastructure > > is such a completely different

Re: RFC: adding pytest as a supported test framework

2024-06-15 Thread Jelte Fennema-Nio
On Sat, 15 Jun 2024 at 19:27, Robert Haas wrote: > This surprises me. I agree that the current state of affairs is kind > of annoying, but the contents of regress_log_whatever are usually > quite long. Printing all of that out to standard output seems like > it's just going to flood the terminal

Re: RFC: adding pytest as a supported test framework

2024-06-15 Thread Jelte Fennema-Nio
On Sat, 15 Jun 2024 at 16:45, Andrew Dunstan wrote: > I see the fact that we stash the output in a file as a feature. Without > it, capturing failure information in the buildfarm client would be more > difficult, especially if there are multiple failures. So this is > actually something I think

Re: RFC: adding pytest as a supported test framework

2024-06-14 Thread Jelte Fennema-Nio
On Fri, 14 Jun 2024 at 17:49, Tom Lane wrote: > But what I'd really like to see is some comparison of the > language-provided testing facilities that we're proposing > to depend on. Why is pytest (or whatever) better than Test::More? Some advantages of pytest over Test::More: 1. It's much

Re: RFC: adding pytest as a supported test framework

2024-06-14 Thread Jelte Fennema-Nio
On Fri, 14 Jun 2024 at 22:33, Greg Sabino Mullane wrote: > I am not happy with the state of Perl, as it has made some MAJOR missteps > along the way, particularly in the last 5 years. But can we dispel this > strawman? There is a difference between "unpopular" and "unmaintained". The > latest

Re: RFC: adding pytest as a supported test framework

2024-06-13 Thread Jelte Fennema-Nio
On Thu, 13 Jun 2024 at 23:35, Andrew Dunstan wrote: > Clearly there are areas we can improve, and we need to be somewhat more > proactive about it. To follow that great suggestion, I updated meson wiki[1] after I realized some of the major gripes I had with the Perl tap test output were not

Re: RFC: adding pytest as a supported test framework

2024-06-13 Thread Jelte Fennema-Nio
On Thu, 13 Jun 2024 at 20:11, Robert Haas wrote: > > But Perl is at the next level of unmaintained infrastructure. It is > > actually clear how you can contribute to it, but still no new > > community members actually want to contribute to it. Also, it's not > > only unmaintained by us but it's

Re: RFC: adding pytest as a supported test framework

2024-06-13 Thread Jelte Fennema-Nio
On Thu, 13 Jun 2024 at 15:38, Robert Haas wrote: > For whatever reason, it seems like every piece of > infrastructure that the PostgreSQL community has suffers from severe > neglect. Literally everything I know of either has one or maybe two > very senior hackers maintaining it, or no maintainer

Re: RFC: adding pytest as a supported test framework

2024-06-13 Thread Jelte Fennema-Nio
On Thu, 13 Jun 2024 at 17:19, Tom Lane wrote: > I wonder if we should be checking out some of the other newer > languages that were mentioned upthread. If this is actually something that we want to seriously evaluate, I think that's a significant effort. And I think the people that want a

Re: RFC: adding pytest as a supported test framework

2024-06-12 Thread Jelte Fennema-Nio
On Wed, 12 Jun 2024 at 21:07, Robert Haas wrote: > Yeah, I don't think depending on psycopg2 is practical at all. We can > either shell out to psql like we do now, or we can use something like > CFFI. Quick clarification I meant psycopg3, not psycopg2. And I'd very much like to avoid using psql

Re: RFC: adding pytest as a supported test framework

2024-06-12 Thread Jelte Fennema-Nio
ome dynamic lookup code ourselves anyway, we might as well have all the dynamic lookup code in our repo to avoid dependencies. On Wed, 12 Jun 2024 at 18:46, Daniele Varrazzo wrote: > > On Wed, 12 Jun 2024 at 18:08, Jelte Fennema-Nio wrote: > > > > On Wed, 12 Jun 2024 at 17:50,

Re: RFC: adding pytest as a supported test framework

2024-06-12 Thread Jelte Fennema-Nio
On Wed, 12 Jun 2024 at 17:50, Andres Freund wrote: > > The OAuth pytest suite makes extensive use of > > - psycopg, to easily drive libpq; > > That's probably not going to fly. It introduces painful circular dependencies > between building postgres (for libpq), building psycopg (requiring libpq)

Re: RFC: adding pytest as a supported test framework

2024-06-12 Thread Jelte Fennema-Nio
aces myself too. But honestly that's pretty much a tabs vs spaces discussion. [1]: https://pypi.org/project/pytest-xdist/ On Wed, 12 Jun 2024 at 15:49, FWS Neil wrote: > > > > On Jun 12, 2024, at 6:40 AM, Jelte Fennema-Nio wrote: > > > > I think C#, Java, Go, Rust, Kot

Re: Proposal: Document ABI Compatibility

2024-06-12 Thread Jelte Fennema-Nio
On Wed, 12 Jun 2024 at 14:44, Peter Eisentraut wrote: > I think since around 6 years ago we have been much more vigilant about > avoiding ABI breaks. So if there aren't any more recent examples of > breakage, then maybe that was ultimately successful, and the upshot is, > continue to be vigilant

Re: RFC: adding pytest as a supported test framework

2024-06-12 Thread Jelte Fennema-Nio
On Wed, 12 Jun 2024 at 01:48, Noah Misch wrote: > If we're going to test in a non-Perl language, I'd pick C over Python. > > We'd need a library like today's Perl > PostgreSQL::Test to make C-language tests nice, but the same would apply to > any new language. P.P.S. We already write tests in

Re: RFC: adding pytest as a supported test framework

2024-06-12 Thread Jelte Fennema-Nio
On Wed, 12 Jun 2024 at 01:48, Noah Misch wrote: > If we're going to test in a non-Perl language, I'd pick C over Python. There > would be several other unlikely-community-choice languages I'd pick over > Python (C#, Java, C++). My main goals of this thread are: 1. Allowing people to quickly

Re: Re: Re: Add support to TLS 1.3 cipher suites and curves lists

2024-06-12 Thread Jelte Fennema-Nio
On Wed, 12 Jun 2024 at 04:32, Erica Zhang wrote: > There are certain government, financial and other enterprise organizations > that have very strict requirements about the encrypted communication and more > specifically about fine grained params like the TLS ciphers and curves that > they

Re: Add support to TLS 1.3 cipher suites and curves lists

2024-06-12 Thread Jelte Fennema-Nio
On Mon, 10 Jun 2024 at 12:31, Daniel Gustafsson wrote: > Regarding the ciphersuites portion of the patch. I'm not particularly > thrilled > about having a GUC for TLSv1.2 ciphers and one for TLSv1.3 ciphersuites, users > not all that familiar with TLS will likely find it confusing to figure out

Re: Partial aggregates pushdown

2024-06-12 Thread Jelte Fennema-Nio
On Wed, 12 Jun 2024 at 07:27, fujii.y...@df.mitsubishielectric.co.jp wrote: > Could you please clarify what you mean? > Are you referring to: > Option 1: Modifying existing aggregate functions to minimize the use of > internal state values. > Option 2: Not supporting the push down of partial

Re: Partial aggregates pushdown

2024-06-11 Thread Jelte Fennema-Nio
On Tue, 11 Jun 2024 at 13:40, fujii.y...@df.mitsubishielectric.co.jp wrote: > > From: Bruce Momjian > > So, we need import/export text representation for the partial aggregate > > mode for these eight, and call the base data type > > text import/export functions for the zero ones when in this

Re: Addressing SECURITY DEFINER Function Vulnerabilities in PostgreSQL Extensions

2024-06-11 Thread Jelte Fennema-Nio
On Tue, 11 Jun 2024 at 11:54, Ashutosh Sharma wrote: > 1) Extends the CREATE EXTENSION command to support a new option, SET > SEARCH_PATH. I don't think it makes sense to add such an option to CREATE EXTENSION. I feel like such a thing should be part of the extension control file instead. That

Re: RFC: adding pytest as a supported test framework

2024-06-10 Thread Jelte Fennema-Nio
On Mon, 10 Jun 2024 at 22:47, Andrew Dunstan wrote: > As for what up and coming developers learn, they mostly don't learn C either, > and that's far more critical to what we do. I think many up and coming devs have at least touched C somewhere (e.g. in university). And because it's more

Re: RFC: adding pytest as a supported test framework

2024-06-10 Thread Jelte Fennema-Nio
On Mon, 10 Jun 2024 at 20:46, Jacob Champion wrote: > For the v18 cycle, I would like to try to get pytest [1] in as a > supported test driver, in addition to the current offerings. Huge +1 from me (but I'm definitely biased here) > Thoughts? Suggestions? I think the most important thing is

Re: Addressing SECURITY DEFINER Function Vulnerabilities in PostgreSQL Extensions

2024-06-06 Thread Jelte Fennema-Nio
On Thu, 6 Jun 2024 at 20:10, Isaac Morland wrote: > > On Thu, 6 Jun 2024 at 12:53, Jeff Davis wrote: > >> >> > I didn't get you completely here. w.r.t extensions how will this have >> > an impact if we set the search_path for definer functions. >> >> If we only set the search path for SECURITY

Re: ssl tests fail due to TCP port conflict

2024-06-06 Thread Jelte Fennema-Nio
On Wed, 5 Jun 2024 at 23:37, Tom Lane wrote: > > Andrew Dunstan writes: > > On 2024-06-05 We 16:00, Alexander Lakhin wrote: > >> That is, psql from the test instance 001_ssltests_34 opened a > >> connection to > >> the test server with the client port 50072 and it made using the port by > >> the

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-06-06 Thread Jelte Fennema-Nio
On Thu, 6 Jun 2024 at 18:01, Robert Haas wrote: > As I see it, the issue here is whether the default value would ever be > different from the latest value. If not, then using blank to mean the > latest seems fine, but if so, then I'd expect blank to mean the > default version and latest to mean

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-06-06 Thread Jelte Fennema-Nio
On Thu, 6 Jun 2024 at 03:03, Robert Haas wrote: > This makes some sense to me. I don't think that I believe > max_protocol_version=latest is future-proof: just because I want to > opt into this round of new features doesn't mean I feel the same way > about the next round. But it may still be a

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-06-05 Thread Jelte Fennema-Nio
On Wed, 5 Jun 2024 at 22:48, Robert Haas wrote: > I agree that we need such a mechanism, but if the driving feature is > cancel-key length, I expect that opt-in isn't going to work very well. > Opt-in seems like it would work well with compression or transparent > column encryption, but few users

Re: [multithreading] extension compatibility

2024-06-05 Thread Jelte Fennema-Nio
On Wed, 5 Jun 2024 at 22:05, Robert Haas wrote: > The attached patch is a sketch of one possible approach: PostgreSQL > signals whether it is multithreaded by defining or not defining > PG_MULTITHREADING in pg_config_manual.h, and an extension signals > thread-readiness by defining

Re: Extension security improvement: Add support for extensions with an owned schema

2024-06-05 Thread Jelte Fennema-Nio
On Wed, 5 Jun 2024 at 19:53, Jeff Davis wrote: > Is this orthogonal to relocatability? It's fairly orthogonal, but it has some impact on relocatability: You can only relocate to a schema name that does not exist yet (while currently you can only relocate to a schema that already exists). This is

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-06-05 Thread Jelte Fennema-Nio
On Wed, 5 Jun 2024 at 17:12, Robert Haas wrote: > Well, hang on. The discussion on that thread suggests that this is > only going to break connections to servers that don't have > NegotiateProtocolVersion. Yes, correct. > What > I really don't want is for v18 to break connections to v17

libpq: Trace StartupMessage/SSLRequest/GSSENCRequest correctly

2024-06-05 Thread Jelte Fennema-Nio
ace From 6fe11a1656e3bc85608e3d883848e42cd765e553 Mon Sep 17 00:00:00 2001 From: Jelte Fennema-Nio Date: Wed, 5 Jun 2024 12:07:09 +0200 Subject: [PATCH v1] libpq: Trace StartupMessage/SSLRequest/GSSENCRequest correctly These messages would previously be logged in the tracing output as: Unknown message: length %d With thi

Extension security improvement: Add support for extensions with an owned schema

2024-06-01 Thread Jelte Fennema-Nio
that function. From cca2fc5a820822a29d9bc3b810a811adb1d081e1 Mon Sep 17 00:00:00 2001 From: Jelte Fennema-Nio Date: Fri, 31 May 2024 02:04:31 -0700 Subject: [PATCH v1] Add support for extensions with an owned schema Writing the sql migration scripts that are run by CREATE EXTENSION and ALTER EXTENSION UPDATE are

Re: In-placre persistance change of a relation

2024-05-28 Thread Jelte Fennema-Nio
On Fri, 24 May 2024 at 00:09, Kyotaro Horiguchi wrote: > Along with rebasing, I changed the interface of XLogFsyncFile() to > return a boolean instead of an error message. Two notes after looking at this quickly during the advanced patch feedback session: 1. I would maybe split 0003 into two

Re: Fix calloc check if oom (PQcancelCreate)

2024-05-27 Thread Jelte Fennema-Nio
On Mon, 27 May 2024 at 18:16, Ranier Vilela wrote: > Is it mandatory to call PQcancelFinish in case PQcancelCreate fails? Yes, see the following line in the docs: Note that when PQcancelCreate returns a non-null pointer, you must call PQcancelFinish when you are finished with it, in order to

Re: Fix calloc check if oom (PQcancelCreate)

2024-05-27 Thread Jelte Fennema-Nio
On Mon, 27 May 2024 at 16:06, Ranier Vilela wrote: > Em seg., 27 de mai. de 2024 às 10:23, Daniel Gustafsson > escreveu: >> > On 27 May 2024, at 14:25, Ranier Vilela wrote: >> > I think that commit 61461a3, left some oversight. >> > The function *PQcancelCreate* fails in check, >> > return of

Re: DISCARD ALL does not force re-planning of plpgsql functions/procedures

2024-05-27 Thread Jelte Fennema-Nio
On Sun, 26 May 2024 at 19:39, Tom Lane wrote: > Hm, should it be? That's hard-won knowledge, and I'm not sure there > is a good reason to believe it's no longer applicable. Okay, so I looked into this a bit more and there's a similar case here that's imho very clearly doing something

Re: DISCARD ALL does not force re-planning of plpgsql functions/procedures

2024-05-26 Thread Jelte Fennema-Nio
On Sun, May 26, 2024, 12:26 Jelte Fennema-Nio wrote: > DISCARD PLANS should probably forget about it though indeed. > DISCARD PLANS should probably **not** forget about it > > Note that any change in behavior there would affect prepared > > statements in general, not only pl

Re: DISCARD ALL does not force re-planning of plpgsql functions/procedures

2024-05-26 Thread Jelte Fennema-Nio
On Sun, 26 May 2024 at 19:39, Tom Lane wrote: > Hm, should it be? That's hard-won knowledge, and I'm not sure there > is a good reason to believe it's no longer applicable. I think for DISCARD ALL it would probably make sense to forget this knowledge . Since that is advertised as "reset the

DISCARD ALL does not force re-planning of plpgsql functions/procedures

2024-05-26 Thread Jelte Fennema-Nio
I got a report on the PgBouncer repo[1] that running DISCARD ALL was not sufficient between connection handoffs to force replanning of stored procedures. Turns out that while DISCARD AL and DISCARD PLAN reset the plan cache they do not reset the num_custom_plans fields of the existing PlanSources.

Re: Volatile write caches on macOS and Windows, redux

2024-05-25 Thread Jelte Fennema-Nio
On Tue, 18 Jul 2023 at 05:29, Thomas Munro wrote: > It's possible that fcntl(F_FULLFSYNC) might fail with ENOSUPP or other > errors in obscure cases (eg unusual file systems). In that case, you > could manually lower fsync to just "on" and do your own research on > whether power loss can toast

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-25 Thread Jelte Fennema-Nio
(pressed send to early) On Sat, 25 May 2024 at 12:39, Jelte Fennema-Nio wrote: > But again if I'm alone in this, then I don't ... mind budging on this to move this decision along. Using _pq_.xxx parameters for all protocol changes would totally be acceptable to me.

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-25 Thread Jelte Fennema-Nio
On Thu, 23 May 2024 at 20:12, Tom Lane wrote: > > Jelte Fennema-Nio writes: > > On Fri, 17 May 2024 at 21:24, Robert Haas wrote: > >> Perhaps these are all minority positions, but I can't tell you what > >> everyone thinks, only what I think. > > > I'd

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-25 Thread Jelte Fennema-Nio
On Thu, 23 May 2024 at 20:40, Tom Lane wrote: > > Jacob Champion writes: > > Would it be good to expand on that idea of criticality? IIRC one of > > Jelte's complaints earlier was that middleware has to know all the > > extension types anyway, to be able to figure out whether it has to do > >

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-24 Thread Jelte Fennema-Nio
On Fri, 24 May 2024 at 15:28, Robert Haas wrote: > > On Thu, May 23, 2024 at 4:00 PM Tom Lane wrote: > > I don't recall exactly what I thought earlier, but now I think we'd > > be better off with separate infrastructure. guc.c is unduly complex > > already. Perhaps there are bits of it that

Re: libpq compression (part 3)

2024-05-21 Thread Jelte Fennema-Nio
On Mon, 20 May 2024 at 21:42, Jacob Champion wrote: > As Andrey points out, there was prior work done that started to take > this into account. I haven't reviewed it to see how good it is -- and > I think there are probably many use cases in which queries and tables > contain both private and

Re: Multiple startup messages over the same connection

2024-05-19 Thread Jelte Fennema-Nio
On Sat, 18 May 2024 at 23:10, Vladimir Churyukin wrote: > I guess the process of passing control from child processes to the parent > could be a bit tricky for that one, but doable? > Is there anything I'm missing that can be a no-go for this? One seriously difficult/possibly impossible thing

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 21:24, Robert Haas wrote: > I think the > fear that we're going to run into cases where such complex handshaking > is necessary is a major reason why I'm afraid of Jelte's ideas about > ParameterSet: it seems much more opinionated to me than he seems to > think it is. I

Re: libpq compression (part 3)

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 23:10, Jacob Champion wrote: > We're talking about a transport-level option, though -- I thought the > proposal enabled compression before authentication completed? Or has > that changed? I think it would make sense to only compress messages after authentication has

Re: libpq compression (part 3)

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 23:40, Robert Haas wrote: > To be clear, I am not arguing that it should be the receiver's choice. > I'm arguing it should be the client's choice, which means the client > decides what it sends and also tells the server what to send to it. > I'm open to counter-arguments,

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 17:59, Robert Haas wrote: > If they > get attention, it's much more likely to be because somebody saw their > email and wrote back than it is to be because somebody went through > the CommitFest and found their entry and was like "oh, I should review > this". I think this

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 14:19, Joe Conway wrote: > > On 5/16/24 22:26, Robert Haas wrote: > > For example, imagine that the CommitFest is FORCIBLY empty > > until a week before it starts. You can still register patches in the > > system generally, but that just means they get CI runs, not that > >

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 13:39, Robert Haas wrote: > > On Fri, May 17, 2024 at 7:11 AM Tomas Vondra > wrote: > > Yeah, I 100% agree with this. If a patch bitrots and no one cares enough > > to rebase it once in a while, then sure - it's probably fine to mark it > > RwF. But forcing all

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 11:02, Jelte Fennema-Nio wrote: > > On Fri, 17 May 2024 at 10:47, Daniel Gustafsson wrote: > > One way to ensure we capture detail could be if the system would send an > > automated email to the thread summarizing the entry when it's marked

  1   2   3   4   5   6   >