Re: Cleaning up threading code

Hi, On 2023-06-10 14:23:52 +1200, Thomas Munro wrote: > 1. We should completely remove --disable-thread-safety and the > various !defined(ENABLE_THREAD_SAFETY) code paths. Yes, please! > 2. I don't like the way we have to deal with POSIX vs Windows at > every site where we use threads, and

Re: Cleaning up nbtree after logical decoding on standby work

On Fri, Jun 9, 2023 at 9:40 PM Andres Freund wrote: > I don't think minimizing heaprel being passed around is a worthwhile goal, the > contrary actually: It just makes it painful to use heaprel anywhere, because > it causes precisely these cascading changes of adding/removing the parameter > to a

Re: Cleaning up nbtree after logical decoding on standby work

Hi, On 2023-06-09 12:23:36 -0700, Peter Geoghegan wrote: > On Fri, Jun 9, 2023 at 12:03 PM Andres Freund wrote: > > > My new plan is to commit this tomorrow, since the clear consensus is > > > that we should go ahead with this for 16. > > > > I'm not sure there is that concensus (for me half the

Re: could not extend file "base/5/3501" with FileFallocate(): Interrupted system call

Hi, On 2023-06-06 21:53:00 +0200, Alvaro Herrera wrote: > On 2023-Apr-24, Andres Freund wrote: > > > A prototype of that approach is attached. I pushed the retry handling into > > the > > pg_* routines where applicable. I guess we could add pg_* routines for > > FileFallocate(), FilePrewarm()

Re: pg_stat_io for the startup process

Hi, On 2023-05-22 13:36:42 +0900, Kyotaro Horiguchi wrote: > At Sun, 21 May 2023 15:14:23 -0700, Andres Freund wrote > in > > Hi, > > > > I don't really feel confident we're going to get the timeout approach solid > > enough for something going into the tree around beta 1. > > > > How about

Re: Documentation for building with meson

Hi, On 2023-03-28 12:27:26 -0700, samay sharma wrote: > + > +If you want to build the docs, you can type: > + > +ninja docs > + > + To me 'you can type' sounds odd. To me even just "To build the docs:" would sound better. But the make docs do it "your" way, so I'll just go along with

Re: Use COPY for populating all pgbench tables

On Fri, Jun 9, 2023 at 6:20 PM Gurjeet Singh wrote: > On Fri, Jun 9, 2023 at 5:42 PM Gregory Smith wrote: > > On Fri, Jun 9, 2023 at 1:25 PM Gurjeet Singh wrote: > >> > >> > $ pgbench -i -I dtGvp -s 500 > >> > >> The steps are severely under-documented in pgbench --help output. > > > > I

Re: [PATCH] Missing dep on Catalog.pm in meson rules

Hi, On 2023-06-09 13:58:46 -0500, Tristan Partin wrote: > Patch looks good to me! Thanks for the report Ilmari and the review Tristan! Pushed the fix. Regards, Andres

Re: WL_SOCKET_ACCEPT fairness on Windows

On Thu, May 18, 2023 at 8:53 PM Wei Wang (Fujitsu) wrote: > On Sat, April 1, 2023 at 11:00 AM Thomas Munro wrote: > I tried to test this patch on Windows. And I did cover the new code path > below: > ``` > + /* We have another event to decode. */ > + cur_event =

Add last_commit_lsn to pg_stat_database

I've previously noted in "Add last commit LSN to pg_last_committed_xact()" [1] that it's not possible to monitor how many bytes of WAL behind a logical replication slot is (computing such is obviously trivial for physical slots) because the slot doesn't need to replicate beyond the last commit. In

Re: Fix search_path for all maintenance commands

On Fri, 2023-06-09 at 16:29 -0700, Gurjeet Singh wrote: > I'm not sure if mine is a valid concern, and it has been a long time > since I looked at the search_path's and switching Role's implications > (CVE-2009-4136) so pardon my ignorance. > > It feels a bit late in release cycle to introduce

Re: Use COPY for populating all pgbench tables

On Fri, Jun 9, 2023 at 5:42 PM Gregory Smith wrote: > > On Fri, Jun 9, 2023 at 1:25 PM Gurjeet Singh wrote: >> >> > $ pgbench -i -I dtGvp -s 500 >> >> The steps are severely under-documented in pgbench --help output. > > > I agree it's not easy to find information. I just went through

Re: PG 16 draft release notes ready

On Thu, Jun 8, 2023 at 02:23:33PM +0900, Masahiko Sawada wrote: > Hi, > > On Fri, May 19, 2023 at 5:49 AM Bruce Momjian wrote: > > > > I have completed the first draft of the PG 16 release notes. You can > > see the output here: > > > > https://momjian.us/pgsql_docs/release-16.html > >

Re: Use COPY for populating all pgbench tables

On Fri, Jun 9, 2023 at 1:25 PM Gurjeet Singh wrote: > > $ pgbench -i -I dtGvp -s 500 > > The steps are severely under-documented in pgbench --help output. > I agree it's not easy to find information. I just went through double checking I had the order recently enough to remember what I

Re: Let's make PostgreSQL multi-threaded

On Thu, Jun 8, 2023 at 11:37:00AM +1200, Thomas Munro wrote: > It's old, but this describes the 4 main models and which well known > RDBMSes use them in section 2.3: > > https://dsf.berkeley.edu/papers/fntdb07-architecture.pdf > > TL;DR DB2 is the winner, it can do process-per-connection, >

Re: Let's make PostgreSQL multi-threaded

On Wed, Jun 7, 2023 at 06:38:38PM +0530, Ashutosh Bapat wrote: > With multiple processes, we can use all the available cores (at least > theoretically if all those processes are independent). But is that > guaranteed with single process multi-thread model? Google didn't throw > any definitive

Re: Fix search_path for all maintenance commands

On Fri, Jun 9, 2023 at 2:00 PM Jeff Davis wrote: > > On Thu, 2023-06-08 at 21:55 -0700, Nathan Bossart wrote: > > On Thu, Jun 08, 2023 at 06:08:08PM -0400, Greg Stark wrote: > > > I guess that's pretty narrow and a reasonable thing to desupport. > > > Users could just mark those functions with

add non-option reordering to in-tree getopt_long

While working on 2dcd157, I noticed cfbot failures for Windows due to tests with commands that had non-options specified before options. For example, "createuser myrole -a myadmin" specified a non-option (myrole) before the option/argument pair (-a admin). To get the tests passing for Windows,

Re: Let's make PostgreSQL multi-threaded

Greetings, * Dave Cramer (davecramer@postgres.rocks) wrote: > One thing I can think of is upgrading. AFAIK dump and restore is the only > way to change the on disk format. > Presuming that eventually we will be forced to change the on disk format it > would be nice to be able to do so in a manner

Re: index prefetching

On Thu, Jun 8, 2023 at 11:40 AM Tomas Vondra wrote: > We already do prefetching for bitmap index scans, where the bitmap heap > scan prefetches future pages based on effective_io_concurrency. I'm not > sure why exactly was prefetching implemented only for bitmap scans At the point Greg Stark

Re: Fix search_path for all maintenance commands

On Thu, 2023-06-08 at 21:55 -0700, Nathan Bossart wrote: > On Thu, Jun 08, 2023 at 06:08:08PM -0400, Greg Stark wrote: > > I guess that's pretty narrow and a reasonable thing to desupport. > > Users could just mark those functions with search_path or schema > > qualify the object references in

Re: win32ver data in meson-built postgres.exe

On Thu, Jun 08, 2023 at 01:10:00PM +0200, Magnus Hagander wrote: > On Thu, Jun 8, 2023 at 3:45 AM Noah Misch wrote: > > On Wed, Jun 07, 2023 at 04:47:26PM -0700, Andres Freund wrote: > > > On 2023-06-07 16:14:07 -0700, Noah Misch wrote: > > > > postgres.exe is icon-free. > > > > > > We could

Re: Cleaning up nbtree after logical decoding on standby work

On Fri, Jun 9, 2023 at 12:03 PM Andres Freund wrote: > From what I can tell it's largely consistent with other parameters of the > respective function. E.g. btinsert(), _bt_doinsert() use camelCase for most > parameters, so heapRel fits in. There are a few cases where it's not obvious > what the

Adding a pg_get_owned_sequence function?

Hi hackers, I've always been annoyed by the fact that pg_get_serial_sequence takes the table and returns the sequence as strings rather than regclass. And since identity columns were added, the name is misleading as well (which is even acknowledged in the docs, together with a suggestion for a

Re: Cleaning up nbtree after logical decoding on standby work

Hi, On 2023-06-08 08:50:31 -0700, Peter Geoghegan wrote: > On Thu, Jun 8, 2023 at 7:22 AM Alvaro Herrera wrote: > > IMO this kind of change definitely does not have place in a post-beta1 > > restructuring patch. We rarely indulge in case-fixing exercises at any > > other time, and I don't see

Re: [PATCH] Missing dep on Catalog.pm in meson rules

Patch looks good to me! -- Tristan Partin Neon (https://neon.tech)

Re: [PATCH] Missing dep on Catalog.pm in meson rules

Hi, On 2023-06-02 10:13:44 -0500, Tristan Partin wrote: > On Fri Jun 2, 2023 at 8:47 AM CDT, Andres Freund wrote: > > Hi, > > > > On 2023-06-02 08:10:43 -0500, Tristan Partin wrote: > > > > I wonder if we instead could just make perl output the files it loads > > > > and > > > > handle

Re: Meson build updates

Hi, On 2023-06-09 13:15:27 -0500, Tristan Partin wrote: > On Fri Jun 9, 2023 at 11:41 AM CDT, Andres Freund wrote: > > I like the current version better - depending on the meson version makes it > > easy to find what needs to be removed when we upgrade the meson minimum > > version. > > I think

Re: index prefetching

On Fri, Jun 9, 2023 at 3:45 AM Tomas Vondra wrote: > > What the exact historical timeline is may not be that important. My > > emphasis on ScalarArrayOpExpr is partly due to it being a particularly > > compelling case for both parallel index scan and prefetching, in > > general. There are many

Re: Meson build updates

On Fri Jun 9, 2023 at 11:41 AM CDT, Andres Freund wrote: > Hi, > > On 2023-05-18 10:36:59 -0500, Tristan Partin wrote: > > From b35ecb2c8dcd71608f98af1e0ec19d965099ceab Mon Sep 17 00:00:00 2001 > > From: Tristan Partin > > Date: Wed, 17 May 2023 09:40:02 -0500 > > Subject: [PATCH postgres v1

Re: Use COPY for populating all pgbench tables

On Fri Jun 9, 2023 at 12:25 PM CDT, Gurjeet Singh wrote: > On Fri, Jun 9, 2023 at 6:24 AM Gregory Smith wrote: > > > > Unfortunately there's no simple command line option to change just that one > > thing about how pgbench runs. You have to construct a command line that > > documents each and

Re: win32ver data in meson-built postgres.exe

Hi, On 2023-06-07 18:45:07 -0700, Noah Misch wrote: > On Wed, Jun 07, 2023 at 04:47:26PM -0700, Andres Freund wrote: > > On 2023-06-07 16:14:07 -0700, Noah Misch wrote: > > > A postgres.exe built with meson, ninja, and MSVC lacks the version > > > metadata > > > that postgres.exe gets under

Re: Use COPY for populating all pgbench tables

On Fri, Jun 9, 2023 at 6:24 AM Gregory Smith wrote: > > Unfortunately there's no simple command line option to change just that one > thing about how pgbench runs. You have to construct a command line that > documents each and every step you want instead. You probably just want this > form:

Re: Partial aggregates pushdown

On Tue, Jun 6, 2023 at 03:08:50AM +, fujii.y...@df.mitsubishielectric.co.jp wrote: > > I've seen this message. But introduction of new built-in function will break > > requests to old servers only if this new function is used in the request > > (this > > means that query changes). However,

Re: Meson build updates

Hi, On 2023-05-18 10:36:59 -0500, Tristan Partin wrote: > I took some time to look at the Meson build for Postgres. I contribute > some of the time to Meson, itself. Within this patchset you will find > pretty small changes. Most of the commits are attempting to create more > consistency with the

Re: Views no longer in rangeTabls?

David Steele writes: > Thank you, this was very helpful. I am able to get the expected result > now with: > /* We only care about tables/views and can ignore subqueries, etc. */ > if (!(rte->rtekind == RTE_RELATION || > (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid >

Re: ERROR: wrong varnullingrels (b 3) (expected (b)) for Var 2/1

Richard Guo writes: > On Wed, May 31, 2023 at 10:47 AM Richard Guo wrote: >> When we transform the first form of identity 3 to the second form, we've >> converted Pb*c to Pbc in deconstruct_distribute_oj_quals. But we >> neglect to consider that rel C might be a RTE_SUBQUERY and contains >>

Re: Order changes in PG16 since ICU introduction

On Fri, 2023-06-09 at 14:12 +0200, Daniel Verite wrote: > >  I implemented a compromise where initdb will > >  change C.UTF-8 to the built-in provider > > $ initdb --locale=C.UTF-8 ... > This setup is not what the user has asked for and leads to that kind > of > wrong results: > > $ psql -c

Re: Let's make PostgreSQL multi-threaded

On Fri, 9 Jun 2023 at 17:20, Dave Cramer wrote: > > This is somewhat orthogonal to the topic of threading but relevant to the use > of resources. > > If we are going to undertake some hard problems perhaps we should be looking > at other problems that solve other long term issues before we

Re: Let's make PostgreSQL multi-threaded

This is somewhat orthogonal to the topic of threading but relevant to the use of resources. If we are going to undertake some hard problems perhaps we should be looking at other problems that solve other long term issues before we commit to spending resources on changing the process model. One

Re: Use COPY for populating all pgbench tables

David, I think you should submit this patch standalone. I don't see any reason this shouldn't be reviewed and committed when fully fleshed out. -- Tristan Partin Neon (https://neon.tech)

Re: Use COPY for populating all pgbench tables

On Fri Jun 9, 2023 at 8:24 AM CDT, Gregory Smith wrote: > On Tue, May 23, 2023 at 1:33 PM Tristan Partin wrote: > > > We (Neon) have noticed that pgbench can be quite slow to populate data > > in regard to higher latency connections. Higher scale factors exacerbate > > this problem. Some

Re: Views no longer in rangeTabls?

Hi Amit, On 6/9/23 14:25, Amit Langote wrote: On Fri, Jun 9, 2023 at 17:28 David Steele > wrote: In prior versions of Postgres, views were listed in rangeTabls when ExecutorCheckPerms_hook() was called but in PG16 the views are no longer in this

Re: Meson build updates

Received a review from a Meson maintainer. Here is a v2. -- Tristan Partin Neon (https://neon.tech) From 1ebd8acb56eb0227b09bd7536e1c88ba0059c7ad Mon Sep 17 00:00:00 2001 From: Tristan Partin Date: Tue, 16 May 2023 07:55:03 -0500 Subject: [PATCH v2 01/17] Remove triple-quoted strings

Re: Partial aggregates pushdown

Hi. + An aggregate function, called the partial aggregate function for partial aggregate + that corresponding to the aggregate function, is defined on the primary node and + the postgres_fdw node. Something is clearly wrong here. + When using built-in sharding feature in

Re: Use COPY for populating all pgbench tables

On Tue, May 23, 2023 at 1:33 PM Tristan Partin wrote: > We (Neon) have noticed that pgbench can be quite slow to populate data > in regard to higher latency connections. Higher scale factors exacerbate > this problem. Some employees work on a different continent than the > databases they might

Re: Major pgbench synthetic SELECT workload regression, Ubuntu 23.04+PG15

On Fri, Jun 9, 2023 at 4:06 AM Gurjeet Singh wrote: > There is no mention of perf or similar utilities in pgbench-tools > docs. I'm guessing Linux is the primary platform pgbench-tools gets > used on most. If so, I think it'd be useful to mention these tools and > snippets in there to make

Re: Order changes in PG16 since ICU introduction

Jeff Davis wrote: > I implemented a compromise where initdb will > change C.UTF-8 to the built-in provider This handling of C.UTF-8 would be felt by users as simply broken. With the v10 patches: $ initdb --locale=C.UTF-8 initdb: using locale provider "builtin" for ICU locale

Re: Do we want a hashset type?

On Fri, Jun 9, 2023, at 13:33, jian he wrote: > Hi, I am quite new about C. > The following function I have 3 questions. > 1. 7691,4201, I assume they are just random prime ints? Yes, 7691 and 4201 are likely chosen as random prime numbers. In hash functions, prime numbers are often used to

Re: Do we want a hashset type?

On Fri, Jun 9, 2023 at 6:58 PM Joel Jacobson wrote: > On Thu, Jun 8, 2023, at 12:19, Tomas Vondra wrote: > > Would you be interested in helping with / working on some of that? I > > don't have immediate need for this stuff, so it's not very high on my > > TODO list. > > Sure, I'm willing to

Re: Views no longer in rangeTabls?

Hi David, On Fri, Jun 9, 2023 at 17:28 David Steele wrote: > Hackers, > > While updating pgAudit for PG16 I found the following (from our > perspective) regression. > > In prior versions of Postgres, views were listed in rangeTabls when > ExecutorCheckPerms_hook() was called but in PG16 the

Re: Do we want a hashset type?

On Thu, Jun 8, 2023, at 12:19, Tomas Vondra wrote: > Would you be interested in helping with / working on some of that? I > don't have immediate need for this stuff, so it's not very high on my > TODO list. Sure, I'm willing to help! I've attached a patch that works on some of the items on your

Re: index prefetching

On 6/9/23 01:38, Peter Geoghegan wrote: > On Thu, Jun 8, 2023 at 3:17 PM Tomas Vondra > wrote: >> Normal index scans are an even more interesting case but I'm not >> sure how hard it would be to get that information. It may only be >> convenient to get the blocks from the last leaf

Re: Error in calculating length of encoded base64 string

Tom Lane writes: > Gurjeet Singh writes: >> On Thu, Jun 8, 2023 at 7:35 AM Tom Lane wrote: >>> This bug is very ancient, dating to commit 79d78bb26 which >>> added encode.c. (The other instances were presumably copied >>> from there.) Still, it doesn't quite seem worth back-patching. > >> Is

Re: index prefetching

On 6/9/23 02:06, Andres Freund wrote: > Hi, > > On 2023-06-08 17:40:12 +0200, Tomas Vondra wrote: >> At pgcon unconference I presented a PoC patch adding prefetching for >> indexes, along with some benchmark results demonstrating the (pretty >> significant) benefits etc. The feedback was quite

Re: Introduce WAIT_EVENT_EXTENSION and WAIT_EVENT_BUFFER_PIN

Hi, On 6/9/23 11:20 AM, Masahiro Ikeda wrote: Hi, On 2023-06-09 13:26, Drouvot, Bertrand wrote: Hi, On 6/9/23 1:15 AM, Michael Paquier wrote: On Thu, Jun 08, 2023 at 10:57:55AM +0900, Masahiro Ikeda wrote: (Excuse me for cutting in, and this is not directly related to the thread.) +1. I'm

Re: Support logical replication of DDLs

On Thu, Jun 8, 2023 at 10:02 PM shveta malik wrote: > Please find new set of patches addressing below: > a) Issue mentioned by Wang-san in [1], > b) Comments from Peter given in [2] > c) Comments from Amit given in the last 2 emails. > > [1]: >

Re: Introduce WAIT_EVENT_EXTENSION and WAIT_EVENT_BUFFER_PIN

Hi, On 2023-06-09 13:26, Drouvot, Bertrand wrote: Hi, On 6/9/23 1:15 AM, Michael Paquier wrote: On Thu, Jun 08, 2023 at 10:57:55AM +0900, Masahiro Ikeda wrote: (Excuse me for cutting in, and this is not directly related to the thread.) +1. I'm interested in the feature. This is just a

Re: Inconsistent results with libc sorting on Windows

Juan José Santamaría Flecha wrote: > Just to make sure we are all seeing the same problem, does the attached > patch fix your test? The problem of the random changes in sorting disappears for all libc locales in pg_collation, so this is very promising. However it persists for the

Remove distprep

Per discussion at the unconference[0], I started to write a patch that removes the make distprep target. A more detailed explanation of the rationale is also in the patch. It needs some polishing around the edges, but I wanted to put it out here to get things moving and avoid duplicate work.

Re: Views no longer in rangeTabls?

On 6/9/23 11:28, David Steele wrote: It seems the thing to do here would be to scan permInfos instead, which works fine except that we also need access to rellockmode, which is only included in rangeTabls. We can add a scan of rangeTabls to get rellockmode when needed and we might be better

Re: Remove WindowClause PARTITION BY items belonging to redundant pathkeys

On Fri, Jun 9, 2023 at 8:13 AM David Rowley wrote: > After looking again at nodeWindowAgg.c, I think it might be possible > to do a bit more work and apply this to ORDER BY items too. Without > an ORDER BY clause, all rows in the partition are peers of each other, > and if the ORDER BY column

Views no longer in rangeTabls?

Hackers, While updating pgAudit for PG16 I found the following (from our perspective) regression. In prior versions of Postgres, views were listed in rangeTabls when ExecutorCheckPerms_hook() was called but in PG16 the views are no longer in this list. The permissions have been broken out

Re: Major pgbench synthetic SELECT workload regression, Ubuntu 23.04+PG15

On Fri, Jun 9, 2023 at 12:28 AM Gregory Smith wrote: > > Let me start with the happy ending to this thread: Phew! I'm sure everyone would be relieved to know this was a false alarm. > On Thu, Jun 8, 2023 at 9:21 PM Andres Freund wrote: >> >> You might need to add --no-children to the perf

Re: GTIN14 support for contrib/isn

--- Original Message --- On Thursday, June 8th, 2023 at 5:23 PM, Josef Šimánek wrote: > čt 8. 6. 2023 v 17:20 odesílatel Michael Kefeder m...@multiwave.ch napsal: > > > Am 15.03.19 um 17:27 schrieb Tom Lane: > > > > > Michael Kefeder m...@multiwave.ch writes: > > > > > > > For a

Re: Major pgbench synthetic SELECT workload regression, Ubuntu 23.04+PG15

Let me start with the happy ending to this thread: $ pgbench -S -T 10 -c 32 -j 32 -M prepared -P 1 pgbench pgbench (15.3 (Ubuntu 15.3-1.pgdg23.04+1)) progress: 1.0 s, 1015713.0 tps, lat 0.031 ms stddev 0.007, 0 failed progress: 2.0 s, 1083780.4 tps, lat 0.029 ms stddev 0.007, 0 failed...

Re: Error in calculating length of encoded base64 string

Gurjeet Singh writes: > On Thu, Jun 8, 2023 at 7:35 AM Tom Lane wrote: >> This bug is very ancient, dating to commit 79d78bb26 which >> added encode.c. (The other instances were presumably copied >> from there.) Still, it doesn't quite seem worth back-patching. > Is it worth investing time in

Re: Error in calculating length of encoded base64 string

On Thu, Jun 8, 2023 at 7:35 AM Tom Lane wrote: > > o.tselebrovs...@postgrespro.ru writes: > > While working on an extension I've found an error in how length of > > encoded base64 string is calulated; > > Yeah, I think you're right. It's not of huge significance, because > it just overestimates