Re: pgsql: instr_time: Represent time as an int64 on all platforms
Andres Freund writes: > On 2023-01-23 01:20:54 -0500, Tom Lane wrote: >> Yeah, there was some discussion about that already: >> https://www.postgresql.org/message-id/20230121190303.7xjiwdg3gvb62...@awork3.anarazel.de > I was thinking of starting a starting a separate thread about it - it's > mostly a plpython issue, the fact that my commit caused the compilation > failure is somewhat random. True. It also seems odd to me that per your analysis, we fixed the _POSIX_C_SOURCE conflict on 4 Aug 2011 and then broke it again on 18 Dec 2011, yet nobody has noticed for nigh a dozen years --- there has to be some other element in there. regards, tom lane
Re: pgsql: instr_time: Represent time as an int64 on all platforms
Hi, On 2023-01-23 01:20:54 -0500, Tom Lane wrote: > Michael Paquier writes: > > On Sat, Jan 21, 2023 at 05:25:19AM +, Andres Freund wrote: > >> instr_time: Represent time as an int64 on all platforms > > > hoverfly is unhappy since this went in: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hoverfly=2023-01-23%2005%3A01%3A44 > > Yeah, there was some discussion about that already: > > https://www.postgresql.org/message-id/20230121190303.7xjiwdg3gvb62...@awork3.anarazel.de I was thinking of starting a starting a separate thread about it - it's mostly a plpython issue, the fact that my commit caused the compilation failure is somewhat random. Although I now wonder if we could solve the issue of the compilation failure in a localized way, separately from fixing plpython. There's really no need for execnodes to include instrumentation.h (and thus instr_time). With a forward define of struct Instrumentation and WorkerInstrumentation (and using it in the file), plpython builds just fine with an intentionally broken instr_time.h. > I'm inclined to think that we should fix the plpython code to be rigorous > about including everything else we need before including the Python > headers. That's ugly, but it's not our fault that Python thinks it can > redefine _POSIX_C_SOURCE. Another approach could be to figure out that we ought to define _POSIX_C_SOURCE when building files that involve plpython (e.g. by querying the define during configure). But I'm a bit worried about that breaking assumptions we make - we do define _GNU_SOURCE on linux via CPPFLAGS, and IIRC there are some weird conflicts between the two. Greetings, Andres Freund
Re: pgsql: instr_time: Represent time as an int64 on all platforms
Michael Paquier writes: > On Sat, Jan 21, 2023 at 05:25:19AM +, Andres Freund wrote: >> instr_time: Represent time as an int64 on all platforms > hoverfly is unhappy since this went in: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hoverfly=2023-01-23%2005%3A01%3A44 Yeah, there was some discussion about that already: https://www.postgresql.org/message-id/20230121190303.7xjiwdg3gvb62...@awork3.anarazel.de I'm inclined to think that we should fix the plpython code to be rigorous about including everything else we need before including the Python headers. That's ugly, but it's not our fault that Python thinks it can redefine _POSIX_C_SOURCE. regards, tom lane
Re: pgsql: instr_time: Represent time as an int64 on all platforms
Hi Andres, On Sat, Jan 21, 2023 at 05:25:19AM +, Andres Freund wrote: > instr_time: Represent time as an int64 on all platforms > > Until now we used struct timespec for instr_time on all platforms but > windows. Using struct timespec causes a fair bit of memory (struct timeval is > 16 bytes) and runtime overhead (much more complicated additions). Instead we > can convert the time to nanoseconds in INSTR_TIME_SET_CURRENT(), making the > remaining operations cheaper. > > Representing time as int64 nanoseconds provides sufficient range, ~292 years > relative to a starting point (depending on clock source, relative to the unix > epoch or the system's boot time). That'd not be sufficient for calendar time > stored on disk, but is plenty for runtime interval time measurement. > > On windows instr_time already is represented as cycles. It might make sense to > represent time as cycles on other platforms as well, as using cycle > acquisition instructions like rdtsc directly can reduce the overhead of time > acquisition substantially. This could be done in a fairly localized manner as > the code stands after this commit. > > Because the windows and non-windows paths are now more similar, use a common > set of macros. To make that possible, most of the use of LARGE_INTEGER had to > be removed, which looks nicer anyway. > > To avoid users of the API relying on the integer representation, we wrap the > 64bit integer inside struct struct instr_time. > > Author: Andres Freund > Author: Lukas Fittl hoverfly is unhappy since this went in: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hoverfly=2023-01-23%2005%3A01%3A44 "../../../src/include/portability/instr_time.h", line 116.9: 1506-304 (I) No function prototype given for "clock_gettime". "../../../src/include/portability/instr_time.h", line 116.23: 1506-045 (S) Undeclared identifier CLOCK_REALTIME. : recipe for target 'plpy_cursorobject.o' failed Thanks, -- Michael signature.asc Description: PGP signature
pgsql: pg_walinspect: Add pg_get_wal_fpi_info()
pg_walinspect: Add pg_get_wal_fpi_info() This function is able to extract the full page images from a range of records, specified as of input arguments start_lsn and end_lsn. Like the other functions of this module, an error is returned if using LSNs that do not reflect real system values. All the FPIs stored in a single record are extracted. The module's version is bumped to 1.1. Author: Bharath Rupireddy Reviewed-by: Bertrand Drouvot Discussion: https://postgr.es/m/CALj2ACVCcvzd7WiWvD=6_7nbvvb_r6g0egsxl4f8vosai6s...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/c31cf1c03d01ce86f20bef8c980fe56a257b3b4b Modified Files -- contrib/pg_walinspect/Makefile| 2 +- contrib/pg_walinspect/expected/pg_walinspect.out | 44 - contrib/pg_walinspect/meson.build | 1 + contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql | 24 + contrib/pg_walinspect/pg_walinspect.c | 111 ++ contrib/pg_walinspect/pg_walinspect.control | 2 +- contrib/pg_walinspect/sql/pg_walinspect.sql | 33 ++- doc/src/sgml/pgwalinspect.sgml| 32 +++ 8 files changed, 245 insertions(+), 4 deletions(-)
pgsql: Allow parallel aggregate on string_agg and array_agg
Allow parallel aggregate on string_agg and array_agg This adds combine, serial and deserial functions for the array_agg() and string_agg() aggregate functions, thus allowing these aggregates to partake in partial aggregations. This allows both parallel aggregation to take place when these aggregates are present and also allows additional partition-wise aggregation plan shapes to include plans that require additional aggregation once the partially aggregated results from the partitions have been combined. Author: David Rowley Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane Discussion: https://postgr.es/m/cakjs1f9sx_6gtcvd6tmuznntch0vhbzhx6fzqw17tgvfh-g...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/16fd03e956540d1b47b743f6a84f37c54ac93dd4 Modified Files -- doc/src/sgml/func.sgml | 6 +- src/backend/optimizer/prep/prepagg.c | 28 +- src/backend/parser/parse_agg.c | 37 +- src/backend/utils/adt/array_userfuncs.c | 624 +++ src/backend/utils/adt/arrayfuncs.c | 20 +- src/backend/utils/adt/varlena.c | 207 +- src/include/catalog/catversion.h | 2 +- src/include/catalog/pg_aggregate.dat | 13 +- src/include/catalog/pg_proc.dat | 27 ++ src/include/parser/parse_agg.h | 2 + src/include/utils/array.h| 3 + src/test/regress/expected/aggregates.out | 98 + src/test/regress/sql/aggregates.sql | 62 +++ 13 files changed, 1101 insertions(+), 28 deletions(-)
pgsql: Track logrep apply workers' last start times to avoid useless wa
Track logrep apply workers' last start times to avoid useless waits. Enforce wal_retrieve_retry_interval on a per-subscription basis, rather than globally, and arrange to skip that delay in case of an intentional worker exit. This probably makes little difference in the field, where apply workers wouldn't be restarted often; but it has a significant impact on the runtime of our logical replication regression tests (even though those tests use artificially-small wal_retrieve_retry_interval settings already). Nathan Bossart, with mostly-cosmetic editorialization by me Discussion: https://postgr.es/m/20221122004119.GA132961@nathanxps13 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/5a3a95385bd5a8f1a4fd50545b7efe9338581899 Modified Files -- doc/src/sgml/config.sgml| 4 + doc/src/sgml/monitoring.sgml| 10 ++ src/backend/commands/subscriptioncmds.c | 10 ++ src/backend/replication/logical/launcher.c | 232 ++-- src/backend/replication/logical/tablesync.c | 8 + src/backend/replication/logical/worker.c| 20 +++ src/backend/storage/lmgr/lwlock.c | 4 + src/include/replication/logicallauncher.h | 2 + src/include/storage/lwlock.h| 2 + 9 files changed, 243 insertions(+), 49 deletions(-)
