Re: Skip collecting decoded changes of already-aborted transactions

2023-06-10 Thread Andres Freund
slow if a lot of transactions are in progress concurrently. As soon as the clog buffers are extended all time is spent copying pages from the kernel pagecache. I'd not at all be surprised if this changed causes a substantial slowdown in workloads with lots of small transactions, where most transactio

Re: abi-compliance-checker

2023-06-10 Thread Andres Freund
Hi, On 2023-06-10 12:48:46 -0700, Andres Freund wrote: > > + > > + > > + > > Hm - why is all of this stuff even ending up in the external ABI? It should > all be internal, unless I am missing something? > > I might be looking the wrong way,

Re: abi-compliance-checker

2023-06-10 Thread Andres Freund
06fd8'> > + > + > + > + > + > + > + > + > + > + Hm - why is all of this stuff even ending up in the external ABI? It should all be internal, unless I am missing something? I might be looking the wrong way, but to me it sure looks like none of that ends up being externally visible? Greetings, Andres Freund

Re: Cleaning up threading code

2023-06-09 Thread Andres Freund
nt > +pg_tss_set(pg_tss_t key, void *value) > +{ > +#ifdef WIN32 > + return FlsSetValue(key, value) ? pg_thrd_success : pg_thrd_error; > +#else > + return pg_thrd_maperror(pthread_setspecific(key, value)); > +#endif > +} It's somewhat annoying that this can return errors. > +static inline int > +pg_mtx_init(pg_mtx_t *mutex, int type) I am somewhat confused by the need for these three letter abbreviations... Blaming C11, not you, to be clear. > > -# port needs to be in include path due to pthread-win32.h > +# XXX why do we need this? > libpq_inc = include_directories('.', '../../port') > libpq_c_args = ['-DSO_MAJOR_VERSION=5'] Historically we need it because random places outside of libpq include pthread-win32.h - but pthread-win32.h is in src/port, not in src/include/port. Why on earth it was done this way, I do not know. Greetings, Andres Freund

Re: Cleaning up nbtree after logical decoding on standby work

2023-06-09 Thread Andres Freund
Hi, On 2023-06-09 12:23:36 -0700, Peter Geoghegan wrote: > On Fri, Jun 9, 2023 at 12:03 PM Andres Freund wrote: > > > My new plan is to commit this tomorrow, since the clear consensus is > > > that we should go ahead with this for 16. > > > > I'm not sure the

Re: could not extend file "base/5/3501" with FileFallocate(): Interrupted system call

2023-06-09 Thread Andres Freund
Hi, On 2023-06-06 21:53:00 +0200, Alvaro Herrera wrote: > On 2023-Apr-24, Andres Freund wrote: > > > A prototype of that approach is attached. I pushed the retry handling into > > the > > pg_* routines where applicable. I guess we could add pg_* routines for > >

Re: pg_stat_io for the startup process

2023-06-09 Thread Andres Freund
Hi, On 2023-05-22 13:36:42 +0900, Kyotaro Horiguchi wrote: > At Sun, 21 May 2023 15:14:23 -0700, Andres Freund wrote > in > > Hi, > > > > I don't really feel confident we're going to get the timeout approach solid > > enough for something going into the tree aro

Re: Documentation for building with meson

2023-06-09 Thread Andres Freund
ng of various PostgreSQL features that are not built by default. Most of these are non-default only because they require additional software, as described in Section 17.1." So just merging --without-icu, --without-readline, --without-zlib, --disable-thread-safety, in with the rest doesn't quite seem right. I suspect that the easiest way for that is to just move --disable-atomics, --disable-spinlocks to the developer section and then to leave the anti-features section around for autoconf. Any better idea? Greetings, Andres Freund

Re: [PATCH] Missing dep on Catalog.pm in meson rules

2023-06-09 Thread Andres Freund
Hi, On 2023-06-09 13:58:46 -0500, Tristan Partin wrote: > Patch looks good to me! Thanks for the report Ilmari and the review Tristan! Pushed the fix. Regards, Andres

Re: Cleaning up nbtree after logical decoding on standby work

2023-06-09 Thread Andres Freund
> int sortopt); I think we should continue to provide the table here, even if we don't need it today. Greetings, Andres Freund

Re: [PATCH] Missing dep on Catalog.pm in meson rules

2023-06-09 Thread Andres Freund
Hi, On 2023-06-02 10:13:44 -0500, Tristan Partin wrote: > On Fri Jun 2, 2023 at 8:47 AM CDT, Andres Freund wrote: > > Hi, > > > > On 2023-06-02 08:10:43 -0500, Tristan Partin wrote: > > > > I wonder if we instead could just make perl output the files it

Re: Meson build updates

2023-06-09 Thread Andres Freund
Hi, On 2023-06-09 13:15:27 -0500, Tristan Partin wrote: > On Fri Jun 9, 2023 at 11:41 AM CDT, Andres Freund wrote: > > I like the current version better - depending on the meson version makes it > > easy to find what needs to be removed when we upgrade the meson minimum > >

Re: win32ver data in meson-built postgres.exe

2023-06-09 Thread Andres Freund
Hi, On 2023-06-07 18:45:07 -0700, Noah Misch wrote: > On Wed, Jun 07, 2023 at 04:47:26PM -0700, Andres Freund wrote: > > On 2023-06-07 16:14:07 -0700, Noah Misch wrote: > > > A postgres.exe built with meson, ninja, and MSVC lacks the version > > > metadata > > &

Re: Meson build updates

2023-06-09 Thread Andres Freund
r specifying > paths > > Meson has a built-in way to override paths without polluting project > build options called machine files. I think meson machine files are just about unusable. For one, you can't actually change any of the options without creating a new build dir. For another, it's not something that easily can be done on the commandline. So I really don't want to remove these options. Greetings, Andres Freund

Re: Major pgbench synthetic SELECT workload regression, Ubuntu 23.04+PG15

2023-06-08 Thread Andres Freund
Hi, On 2023-06-08 20:20:18 -0400, Gregory Smith wrote: > On Thu, Jun 8, 2023 at 6:18 PM Andres Freund wrote: > > > Could you get a profile with call graphs? We need to know what leads to all > > those osq_lock calls. > > perf record --call-graph dwarf -a sleep 1 > &g

Re: index prefetching

2023-06-08 Thread Andres Freund
should eliminate most of this, I think. I'm a bit confused by some of these numbers. How can OS-level prefetching lead to massive prefetching in the alread cached case, e.g. in tpch q06 and q08? Unless I missed what "xeon / cached (speedup)" indicates? I think it'd be good to run a performance comparison of the unpatched vs patched cases, with prefetching disabled for both. It's possible that something in the patch caused unintended changes (say spilling during a hashagg, due to larger struct sizes). Greetings, Andres Freund

Re: Major pgbench synthetic SELECT workload regression, Ubuntu 23.04+PG15

2023-06-08 Thread Andres Freund
Hi, On 2023-06-08 15:18:07 -0700, Andres Freund wrote: > E.g. on my workstation (two sockets, 10 cores/20 threads each), with 32 > clients, performance changes back and forth between ~600k and ~850k. Whereas > with 42 clients, it's steadily at 1.1M, with little variance. FWIW, this is w

Re: Major pgbench synthetic SELECT workload regression, Ubuntu 23.04+PG15

2023-06-08 Thread Andres Freund
when the memory controller is enabled. IIRC that could be alleviated to a substantial degree with cgroup.memory=nokmem. Greetings, Andres Freund

Re: Let's make PostgreSQL multi-threaded

2023-06-08 Thread Andres Freund
ferent PCIDs. > The only way to avoid TLB shootdowns on context switches is to have *exactly > the same memory map*. Or, as Robert succinctly shouted, "THREADS". +1 Greetings, Andres Freund

Re: Let's make PostgreSQL multi-threaded

2023-06-08 Thread Andres Freund
he industry have decided isn't going anywhere. It'd be one thing if we had 5 kernel devs that we could deploy to work on this, but we don't. So we have to convince kernel devs employed by others that somehow this is an urgent enough thing that they should work on it. The likely, imo justified, answer is just going to be: Fix your architecture, then we can talk. Greetings, Andres Freund

Re: Let's make PostgreSQL multi-threaded

2023-06-08 Thread Andres Freund
ed via process exit, making the change realistically reviewable, portability are all much harder. Greetings, Andres Freund

Re: Let's make PostgreSQL multi-threaded

2023-06-08 Thread Andres Freund
and they are primarily interested in changing properties of a memory mapping across multiple processes, e.g. making some memory executable and have that reflected in all processes. I don't think this will help us much. Greetings, Andres Freund

Re: Let's make PostgreSQL multi-threaded

2023-06-08 Thread Andres Freund
rcx, __emutls_v.c[rip] add DWORD PTR [rax], 1 call__emutls_get_address add DWORD PTR [rax], 1 add rsp, 40 ret Greetings, Andres Freund

Re: Let's make PostgreSQL multi-threaded

2023-06-08 Thread Andres Freund
Hi, On 2023-06-08 12:15:58 +0200, Hannu Krosing wrote: > On Thu, Jun 8, 2023 at 11:54 AM Hannu Krosing wrote: > > > > On Wed, Jun 7, 2023 at 11:37 PM Andres Freund wrote: > > > > > > Hi, > > > > > > On 2023-06-05 13:40:13 -0400, Jonat

Re: Let's make PostgreSQL multi-threaded

2023-06-08 Thread Andres Freund
On 2023-06-08 14:01:16 +0200, Jose Luis Tallon wrote: > * For "heavyweight" queries, the scalability of "almost independent" > processes w.r.t. NUMA is just _impossible to achieve_ (locality of > reference!) with a pure threaded system. When CPU+mem-bound > (bandwidth-wise), threads add nothing

Re: Let's make PostgreSQL multi-threaded

2023-06-08 Thread Andres Freund
On 2023-06-08 10:33:26 -0400, Greg Stark wrote: > On Wed, 7 Jun 2023 at 18:09, Andres Freund wrote: > > Having the same memory mapping between threads makes allows the > > hardware to share the TLB (on x86 via process context identifiers), which > > isn't realistically po

Re: Cleaning up nbtree after logical decoding on standby work

2023-06-07 Thread Andres Freund
sing camelcase parameters? This part of the change just blows up the size, making it harder to review. 12 files changed, 317 insertions(+), 297 deletions(-) ... Greetings, Andres Freund

Re: win32ver data in meson-built postgres.exe

2023-06-07 Thread Andres Freund
DING', 'exe' > + ] > + > + rcgen_bin_args = rcgen_server_args + [ > '--ICO', pg_ico >] Somehow it seems a bit wrong to derive non-server from server, but ... ;) Greetings, Andres Freund

Re: is pg_log_standby_snapshot() really needed?

2023-06-07 Thread Andres Freund
.63s on my machine. Without it I lost patience after 2 minutes. And the test was only at the start (8 out of 78 subtests). Greetings, Andres Freund

Re: Let's make PostgreSQL multi-threaded

2023-06-07 Thread Andres Freund
ut threading support - which would allow us to get rid of a very significant amount of code and runtime overhead. Greetings, Andres Freund

Re: Let's make PostgreSQL multi-threaded

2023-06-07 Thread Andres Freund
egister. Whereas each individual variable being in TLS makes the job harder for the compiler. Greetings, Andres Freund

Re: Let's make PostgreSQL multi-threaded

2023-06-07 Thread Andres Freund
Hi, On 2023-06-07 23:39:01 +0200, Peter Eisentraut wrote: > On 07.06.23 23:30, Andres Freund wrote: > > Yea, we definitely need the supervisor function in a separate > > process. Presumably that means we need to split off some of the postmaster > > responsibilities - e.g. I

Re: Let's make PostgreSQL multi-threaded

2023-06-07 Thread Andres Freund
ess state is shared. But that's by far outweighed by most debugging / validation tools not understanding the multi-processes-with-shared-shmem model. Greetings, Andres Freund

Re: Let's make PostgreSQL multi-threaded

2023-06-07 Thread Andres Freund
here's plenty of things we can do before, but in the end I think tackling the issues you mention and moving to threads are quite tightly linked. Greetings, Andres Freund

Re: Let's make PostgreSQL multi-threaded

2023-06-07 Thread Andres Freund
up being beneficial even in the process world. A related issue is that we won't get SIGCHLD in the supervisor process anymore. So we'd need to come up with some design for that. Greetings, Andres Freund

Re: v16 fails to build w/ Visual Studio 2015

2023-06-07 Thread Andres Freund
ssue. It seems quite possible we're going to hit such issues in other places. Greetings, Andres Freund

Re: New Table Access Methods for Multi and Single Inserts

2023-06-03 Thread Andres Freund
bool alloc_bistate, bool is_multi) Why have alloc_bistate and options? > +static inline void > +table_insert_end(TableInsertState *state) > +{ > + /* Deallocate bulk insert state here, since it's AM independent. */ > + if (state->bistate) > + FreeBulkInsertState(state->bistate); > + > + state->rel->rd_tableam->tuple_insert_end(state); > +} Seems like the order in here should be swapped? Greetings, Andres Freund

Re: [PATCH] Missing dep on Catalog.pm in meson rules

2023-06-02 Thread Andres Freund
king of generating a depfile from within perl. Something like what you propose doesn't quite seems like a sufficient improvement. > Otherwise, manual as it is in the original patch seems like an alright > compromise for now. Yea. I'm working on a more complete version, also dealing with

Re: [PATCH] Missing dep on Catalog.pm in meson rules

2023-06-02 Thread Andres Freund
erl output the files it loads and handle dependencies automatically that way? But that's more work, so it's probably the right thing to go for the manual path for now. Greetings, Andres Freund

Re: BF animal dikkop reported a failure in 035_standby_logical_decoding

2023-05-30 Thread Andres Freund
um in verbose mode (as suggested by Shi-san) and > depending of its output run the "invalidation" test or: re-launch the vacuum, > re-check the output > and so on.. (n times max). If n is reached, then skip this test. I think the best fix would be to wait for a new snapshot that has a newer horizon, before doing the vacuum full. Greetings, Andres Freund

Re: testing dist tarballs

2023-05-27 Thread Andres Freund
Hi, On 2023-05-26 09:02:33 +0200, Peter Eisentraut wrote: > On 24.05.23 23:24, Andres Freund wrote: > > First thing I noticed that 'make dist' doesn't work in a vpath, failing in a > > somewhat obscure way (likely because in a vpath build the the copy from the > > sourc

Re: Why does pg_bsd_indent need to be installed?

2023-05-27 Thread Andres Freund
Hi, On 2023-05-25 13:05:57 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2023-05-25 09:09:45 -0400, Tom Lane wrote: > >> Peter Eisentraut writes: > >>> Why does pgindent require that pg_bsd_indent be installed in the path? > > > Isn't the situ

Re: Cleaning up nbtree after logical decoding on standby work

2023-05-26 Thread Andres Freund
_bt_getbuf(). We probably should evolve the design in the area, and that will be easier with such a change. Greetings, Andres Freund

Re: Why does pg_bsd_indent need to be installed?

2023-05-25 Thread Andres Freund
n't do that, because the autoconf vpath build copies such directories, which isn't fun). Greetings, Andres Freund

Re: PostgreSQL 16 Beta 1 release announcement draft

2023-05-24 Thread Andres Freund
ing of > data using [`COPY`](https://www.postgresql.org/docs/16/sql-copy.html) up to > a 300%. It also speeds up concurrent loading when not using COPY, just to a lesser degree. But I can't come up with a concise phrasing for that right now... Greetings, Andres Freund

Re: Proposal: Removing 32 bit support starting from PG17++

2023-05-24 Thread Andres Freund
rts 64 bit atomics these days, via LLWP / SCWP. https://s3-eu-west-1.amazonaws.com/downloads-mips/documents/MD00086-2B-MIPS32BIS-AFP-6.06.pdf documents them as having been added to MIPS32 Release 6, from 2014. Greetings, Andres Freund

Re: Proposal: Removing 32 bit support starting from PG17++

2023-05-24 Thread Andres Freund
Hi, On 2023-05-24 19:51:22 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2023-05-24 17:44:36 -0400, Tom Lane wrote: > >> Hmm, can we really expect atomic 8-byte reads on "relevant" 32-bit > >> platforms? I'd be on board with this if so

Re: PostgreSQL 16 Beta 1 release announcement draft

2023-05-24 Thread Andres Freund
Hi, On 2023-05-24 19:57:39 -0400, Jonathan S. Katz wrote: > On 5/24/23 5:28 PM, Andres Freund wrote: > > > > I think the relation extension improvements ought to be mentioned here as > > well? Up to 3x faster concurrent data load with COPY seems practically >

Re: Proposal: Removing 32 bit support starting from PG17++

2023-05-24 Thread Andres Freund
Hi, On 2023-05-24 17:44:36 -0400, Tom Lane wrote: > Andres Freund writes: > > Dropping CPUs without native atomic operations / without a way to do > > tear-free > > 8 byte reads would make several substantial performance improvements easier, > > while not really dro

Re: Atomic ops for unlogged LSN

2023-05-24 Thread Andres Freund
in isolation. > That's not a reason to reject it, though, just something that makes me > curious. I doubt it's a meaningful, if even measurable win. But removing atomic ops and reducing shared memory space isn't a bad thing, even if there's no immediate benefit... Greetings, Andres Freund

Re: Proposal: Removing 32 bit support starting from PG17++

2023-05-24 Thread Andres Freund
bstantial performance improvements easier, while not really dropping any relevant platform. Etc. Greetings, Andres Freund

Re: SyncRepWaitForLSN waits for XLogFlush?

2023-05-24 Thread Andres Freund
ushed. So a backend with such a spurious SyncRepWaitForLSN() would just wait until the LSN is actually flushed and *then* replicated. Any specific reason for that question? Greetings, Andres Freund

Re: PostgreSQL 16 Beta 1 release announcement draft

2023-05-24 Thread Andres Freund
Q-CONNECT-LOAD-BALANCE-HOSTS) > to libpq, the client library for PostgreSQL. I think the relation extension improvements ought to be mentioned here as well? Up to 3x faster concurrent data load with COPY seems practically relevant. Greetings, Andres Freund

testing dist tarballs

2023-05-24 Thread Andres Freund
Hi, On 2023-05-23 14:51:03 -0400, Tom Lane wrote: > Andres Freund writes: > > I guess I need to go and check how long the "release" tarball generation > > takes... > > It's quick except for the documentation-generating steps. Maybe > we could test that part

Re: walsender performance regression due to logical decoding on standby changes

2023-05-24 Thread Andres Freund
Hi, On 2023-05-24 05:53:51 +, Zhijie Hou (Fujitsu) wrote: > On Tuesday, May 23, 2023 1:53 AM Andres Freund wrote: > > On 2023-05-22 12:15:07 +, Zhijie Hou (Fujitsu) wrote: > > > About "a backend doing logical decoding", do you mean the case when a &g

Re: memory leak in trigger handling (since PG12)

2023-05-24 Thread Andres Freund
nt, and > >> more in line with how we do things. But it's not clear to me why we'd > >> end up with the same problem in a few years with what the patch does. > > > > Because it sets up the pattern of manual memory management and continues to > > run the releva

Re: memory leak in trigger handling (since PG12)

2023-05-24 Thread Andres Freund
oing the work in ExecGetAllUpdatedCols() over and over? Unless I am missing something, the result doesn't change across rows. And it doesn't look that cheap to compute, leaving aside the allocation that bms_union() does. It's visible in profiles, not as a top entry, but still. Perhaps the easiest to backpatch fix is to just avoid recomputing the value? But perhaps it'd be just as problmeatic, because callers might modify ExecGetAllUpdatedCols()'s return value in place... Greetings, Andres Freund

Re: memory leak in trigger handling (since PG12)

2023-05-24 Thread Andres Freund
dCols(rootrel, estate), ExecGetUpdatedCols(rootrel, estate)); } Once in ExecPartitionCheckEmitError(), *twice* in ExecConstraints(), ExecWithCheckOptions(). Some of the partitioning stuff has been added in a really myopic way. Greetings, Andres Freund

Re: memory leak in trigger handling (since PG12)

2023-05-24 Thread Andres Freund
oncern. I think it might be reasonable to use something like ExecGetAllUpdatedColsCtx() in the backbranches, and switch to a short-lived context for the trigger invocations in >= 16. Greetings, Andres Freund

Re: memory leak in trigger handling (since PG12)

2023-05-24 Thread Andres Freund
Hi, On 2023-05-23 23:26:42 +0200, Tomas Vondra wrote: > On 5/23/23 19:14, Andres Freund wrote: > > Hi, > > > > On 2023-05-23 18:23:00 +0200, Tomas Vondra wrote: > >> This means that for an UPDATE with triggers, we may end up calling this > >> for each ro

Re: could not extend file "base/5/3501" with FileFallocate(): Interrupted system call

2023-05-23 Thread Andres Freund
Hi, On 2023-05-24 10:56:28 +0900, Kyotaro Horiguchi wrote: > At Wed, 26 Apr 2023 11:37:55 +1200, Thomas Munro > wrote in > > On Tue, Apr 25, 2023 at 12:16 PM Andres Freund wrote: > > > On 2023-04-24 15:32:25 -0700, Andres Freund wrote: > > > >

Re: unnecessary #include "pg_getopt.h"?

2023-05-23 Thread Andres Freund
d in the patch use optarg, opterr etc. - which are declared in pg_getopt.h. Making it reasonable to directly include pg_getopt.h. I don't really see a need to change anything here? Greetings, Andres Freund

Re: pgsql: Move snowball_create.sql creation into perl file

2023-05-23 Thread Andres Freund
ed to go and check how long the "release" tarball generation takes... Greetings, Andres Freund

Re: memory leak in trigger handling (since PG12)

2023-05-23 Thread Andres Freund
Hi, On 2023-05-23 13:28:30 -0400, Tom Lane wrote: > Andres Freund writes: > > Could it help to have a mode where the executor shutdown hook checks how > > much > > memory is allocated in ExecutorState and warns if its too much? > > It'd be very hard to set a limit f

Re: memory leak in trigger handling (since PG12)

2023-05-23 Thread Andres Freund
ise we'll just end up with the same problem in a few years. Greetings, Andres Freund

Re: PG 16 draft release notes ready

2023-05-22 Thread Andres Freund
Hi, On 2023-05-21 22:52:09 -0400, Bruce Momjian wrote: > On Sun, May 21, 2023 at 10:13:41AM -0700, Andres Freund wrote: > > Hi, > > > > Thanks for the release notes! > > > > > > > > > > > > > > > > > Allow more e

Re: PG 16 draft release notes ready

2023-05-22 Thread Andres Freund
this item as a big win, but I don't know how to describe it in a way > that is helpful for the user to know. In doubt the subject of the commit would just work IMO. Greetings, Andres Freund

Re: walsender performance regression due to logical decoding on standby changes

2023-05-22 Thread Andres Freund
ot the details, but IIRC there's some outfuncs (enum?) that intentionally delay releasing locks till transaction commit. Greetings, Andres Freund

Re: pg_stat_io for the startup process

2023-05-21 Thread Andres Freund
lso not accessible, so that's not a problem. It's not the prettiest solution, but I think the simplicity is worth a lot. Greetings, Andres Freund

Re: PG 16 draft release notes ready

2023-05-21 Thread Andres Freund
Hi, On May 21, 2023 11:46:56 AM PDT, "Jonathan S. Katz" wrote: >On 5/21/23 1:13 PM, Andres Freund wrote: > >> >> Looking through the release notes, I didn't see an entry for >> >> commit c6e0fe1f2a08505544c410f613839664eea9eb21 >> Author: Davi

Re: PG 16 draft release notes ready

2023-05-21 Thread Andres Freund
Hi, Thanks for the release notes! > > > > > Allow more efficient addition of multiple heap and index pages (Andres Freund) > > While the case of extending by multiple pages improved the most, even extending by a single page at a time got a good bit more scalable

Re: walsender performance regression due to logical decoding on standby changes

2023-05-21 Thread Andres Freund
so did some minor comment polishing. Greetings, Andres Freund

Re: walsender performance regression due to logical decoding on standby changes

2023-05-21 Thread Andres Freund
oing logical decoding - which can't progress until the startup process wakes the backend up. So I don't think this is promising avenue in the near term. Greetings, Andres Freund

Re: benchmark results comparing versions 15.2 and 16

2023-05-19 Thread Andres Freund
od > news. My testing scripts have been improved so I should be able to finish > the next round of tests in less time. With "yet to see any significant changes" do you mean that the runs are comparable with earlier runs, showing the same regression? Or that the regression vanished? Or ...? Greetings, Andres Freund

Re: Add operator for dividing interval by an interval

2023-05-19 Thread Andres Freund
Hi, On 2023-05-18 17:03:24 -0400, Tom Lane wrote: > Andres Freund writes: > > What about an interval / interval -> double operator that errors out > > whenever > > month is non-zero? As far as I can tell that would always be deterministic. > > We have

Add operator for dividing interval by an interval

2023-05-18 Thread Andres Freund
for IO). What about an interval / interval -> double operator that errors out whenever month is non-zero? As far as I can tell that would always be deterministic. Greetings, Andres Freund

Re: Introduce WAIT_EVENT_EXTENSION and WAIT_EVENT_BUFFER_PIN

2023-05-17 Thread Andres Freund
On 2023-05-17 09:22:19 -0400, Robert Haas wrote: > Adding pgstat_get_wait_extension adds runtime cost for no corresponding > benefit. Having a special case in the code to avoid that seems worthwhile. I don't think that should ever be used in a path where performance is relevant?

Re: issue with meson builds on msys2

2023-05-17 Thread Andres Freund
Hi, On May 17, 2023 2:51:41 PM PDT, Andrew Dunstan wrote: > >On 2023-05-16 Tu 17:52, Andres Freund wrote: >> >>> I suppose the alternative would be to change the way the buildfarm calls >>> pg_ctl stop. Do you have a concrete suggestion for that? >> The easi

Re: walsender performance regression due to logical decoding on standby changes

2023-05-17 Thread Andres Freund
Hi, On 2023-05-17 12:53:15 -0700, Andres Freund wrote: > I'll try to come up with a benchmark without the issues I pointed out in > https://postgr.es/m/20230517194331.ficfy5brpfq5lrmz%40awork3.anarazel.de Here we go: setup: create primary SELECT pg_create_physical_replication_slot('r

Re: walsender performance regression due to logical decoding on standby changes

2023-05-17 Thread Andres Freund
entSetWait() can be replaced with its CV's > + * counterpart. I don't really understand that XXX - the potential bright future would be to add support for CVs into wait event sets, not to replace WES with a CV? Greetings, Andres Freund

Re: walsender performance regression due to logical decoding on standby changes

2023-05-17 Thread Andres Freund
quot;original" cost of the function. gprof style instrumentation is good for things like code coverage, but for performance evaluation it's normally better to use a sampling profiler like perf. That also causes slowdowns, but largely only in places that already take up substantial execution time. Greetings, Andres Freund

Re: walsender performance regression due to logical decoding on standby changes

2023-05-17 Thread Andres Freund
Hi, On 2023-05-10 08:39:08 +0200, Drouvot, Bertrand wrote: > On 5/9/23 11:00 PM, Andres Freund wrote: > > Hi, > > > > On 2023-05-09 13:38:24 -0700, Jeff Davis wrote: > > > On Tue, 2023-05-09 at 12:02 -0700, Andres Freund wrote: > > > > I don't

Re: pg_stat_io not tracking smgrwriteback() is confusing

2023-05-17 Thread Andres Freund
unds like it's pretty close to being ready. Thanks for the nudge. I just pushed the changes, with some very minor changes (a newline, slight changes in commit messages). I'll go and mark the item as closed. Greetings, Andres Freund

Re: smgrzeroextend clarification

2023-05-16 Thread Andres Freund
Hi, On 2023-05-16 20:40:01 +0200, Peter Eisentraut wrote: > On 10.05.23 20:10, Andres Freund wrote: > > > So if you want to understand what smgrzeroextend() does, you need to > > > mentally combine the documentation of three different functions. Could we > > >

Re: WL_SOCKET_ACCEPT fairness on Windows

2023-05-16 Thread Andres Freund
Hi, On 2023-05-17 08:41:24 +1200, Thomas Munro wrote: > Yeah. No one seems to think this is worth worrying about (please > speak up if you do). +1 - we have much bigger fish to fry IMO. Greetings, Andres Freund

Re: issue with meson builds on msys2

2023-05-16 Thread Andres Freund
; I suppose the alternative would be to change the way the buildfarm calls > pg_ctl stop. Do you have a concrete suggestion for that? The easiest fix is to redirect stdin to /dev/null (or some file, if that's easier to do portably) - that should fix the problem entirely, without needing IPC::Run. Greetings, Andres Freund

Re: Introduce WAIT_EVENT_EXTENSION and WAIT_EVENT_BUFFER_PIN

2023-05-15 Thread Andres Freund
Hi, On 2023-05-16 09:38:54 +0900, Michael Paquier wrote: > On Mon, May 15, 2023 at 05:17:16PM -0700, Andres Freund wrote: > > IMO the submission should include why automating requires these changes > > (yours > > doesn't really either). I can probably figure it

Re: benchmark results comparing versions 15.2 and 16

2023-05-15 Thread Andres Freund
r performance when forcing things to be colocated. Greetings, Andres Freund

Re: Introduce WAIT_EVENT_EXTENSION and WAIT_EVENT_BUFFER_PIN

2023-05-15 Thread Andres Freund
Hi, On 2023-05-16 07:30:54 +0900, Michael Paquier wrote: > On Mon, May 15, 2023 at 11:29:56AM -0700, Andres Freund wrote: > > Without an explanation for why this change is needed for [1], it's hard to > > give useful feedback... > > The point is to integrate the wait event c

Re: benchmark results comparing versions 15.2 and 16

2023-05-15 Thread Andres Freund
Hi, On 2023-05-15 14:20:24 +0900, Michael Paquier wrote: > On Thu, May 11, 2023 at 01:28:40PM +0900, Michael Paquier wrote: > > On Tue, May 09, 2023 at 09:48:24AM -0700, Andres Freund wrote: > >> On 2023-05-08 12:11:17 -0700, Andres Freund wrote: > >>> I can reproduc

Re: issue with meson builds on msys2

2023-05-15 Thread Andres Freund
Hi, On 2023-05-15 15:30:28 -0700, Andres Freund wrote: > As soon as either the pg_ctl for the start, or the whole bash invocation, has > stdin redirected, the problem vanishes. For a moment I thought this could be related to InheritStdHandles() - but no, it doesn't make a difference. T

Re: issue with meson builds on msys2

2023-05-15 Thread Andres Freund
Hi, On 2023-05-15 13:13:26 -0700, Andres Freund wrote: > It wouldn't really - the echo $? inside the system() would report the > error. Which it doesn't - note the "0" in the second output. Ah. Interesting. Part of the issue is perl (or msys?) swalling some error details.

Re: issue with meson builds on msys2

2023-05-15 Thread Andres Freund
Hi, On 2023-05-15 16:01:39 -0400, Andrew Dunstan wrote: > On 2023-05-15 Mo 15:38, Andres Freund wrote: > > Hi, > > > > On 2023-05-05 07:08:39 -0400, Andrew Dunstan wrote: > > > If you want to play I can arrange access. > > Andrew did - thanks! > > >

Re: issue with meson builds on msys2

2023-05-15 Thread Andres Freund
uff that InheritstdHandles() does? Andrew, is it ok if modify pg_ctl.c and rebuild? I don't know how "detached" from the actual buildfarm animal the system you gave me access to is... Greetings, Andres Freund

Re: Introduce WAIT_EVENT_EXTENSION and WAIT_EVENT_BUFFER_PIN

2023-05-15 Thread Andres Freund
ww.postgresql.org/message-id/flat/77a86b3a-c4a8-5f5d-69b9-d70bbf2e9...@gmail.com > > Looking forward to your feedback, Without an explanation for why this change is needed for [1], it's hard to give useful feedback... Greetings, Andres Freund

Re: smgrzeroextend clarification

2023-05-12 Thread Andres Freund
Hi, On May 12, 2023 11:36:23 AM PDT, Thomas Munro wrote: >Just a thought: should RelationCopyStorageUsingBuffer(), the new code >used by CREATE DATABASE with the default strategy WAL_LOG, use the >newer interface so that it creates fully allocated files instead of >sparse ones? I played with

Re: smgrzeroextend clarification

2023-05-12 Thread Andres Freund
Hi, On May 11, 2023 2:37:00 AM PDT, Peter Eisentraut wrote: >On 10.05.23 20:10, Andres Freund wrote: >>> Moreover, the text "except the relation can be extended by multiple blocks >>> at once and the added blocks will be filled with zeroes" doesn't make much

Re: smgrzeroextend clarification

2023-05-10 Thread Andres Freund
ust does it by writing data. The FileFallocate() path in smgrzeroextend() tries to avoid writing data if extending by sufficient blocks - not having dirty data in the kernel page cache can substantially reduce the IO usage. Whereas the FileZero() path just optimizes the number of syscalls (and cache misses etc). Greetings, Andres Freund

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-05-10 Thread Andres Freund
Hi, On 2023-05-10 17:44:07 +0200, Peter Eisentraut wrote: > On 12.03.23 00:41, Andres Freund wrote: > > Hi, > > > > On 2023-03-11 15:34:55 -0800, Mark Dilger wrote: > > > > On Mar 11, 2023, at 3:22 PM, Andres Freund wrote: > > > > > > &

Re: walsender performance regression due to logical decoding on standby changes

2023-05-09 Thread Andres Freund
Hi, On 2023-05-09 13:38:24 -0700, Jeff Davis wrote: > On Tue, 2023-05-09 at 12:02 -0700, Andres Freund wrote: > > I don't think the approach of not having any sort of "registry" of > > whether > > anybody is waiting for the replay position to be updated is &

<    5   6   7   8   9   10   11   12   13   14   >