On Mon, Oct 5, 2026 at 6:24 PM Heikki Linnakangas <[email protected]> wrote:
> I started to look at these, starting from this patch:
Hi Heikki, thanks for taking a look !
> > 0004 increases SINK_BUFFER_LENGTH from its current 32 kB. Profiling the
> > server side shows a huge number of small pread() calls coming from
> > basebackup_read_file(), which is surprising given that elsewhere in the tree
> > (backend/storage/buffer/README) we already mention that a 256 kB ring for
> > seq.
> > scans scans is used because it fits comfortably in L2 cache. On my laptop,
> > with no SSL, no checksum generation or verification, a hot filesystem cache,
> > and a client blackhole to eliminate client I/O, a 10 GB backup over loopback
> > runs at:
> > 3.1 GB/s with the 32 kB buffer,
> > 5.1 GB/s at 128 kB,
> > 5.5 GB/s at 256 kB,
> > 6.0 GB/s at 1 MB
> > (those are average of five runs).
>
> Cool
>
> > The win comes from letting pread()
> > swallow much larger chunks of each segment in one go: per-core L2 caches are
> > 1-2 MB even on laptops these days, syscalls have become more expensive since
> > the Spectre/Meltdown mitigations, and the kernel's default readahead is
> > already in the 128-512 kB range, so there's little reason to trickle the
> > data
> > through 32 kB at a time.
>
> While we're at it, we should probably allocate the buffer with
> palloc_aligned(PG_IO_ALIGN_SIZE); a pread() into an aligned buffer
> should be a little faster.
I've attached patchset with 0004 modified that has this adjustement, but I
haven't quantified the gain for this alone though.
BTW: I've added discussion link and other tags, fixed some minor types in to all
commitmsgs too.
> I wonder about this in read_file_data_into_buffer():
>
> > cnt = basebackup_read_file(fd, sink->bbs_buffer,
> >
> > Min(sink->bbs_buffer_length, length),
> > offset,
> > readfilename, true);
> >
> > /* Can't verify checksums if read length is not a multiple of BLCKSZ.
> > */
> > if (!verify_checksum || (cnt % BLCKSZ) != 0)
> > return cnt;
>
> It doesn't seem right to assume that the count must be a multiple of
> BLCKSZ, especially if we use a larger buffer. Should we do something
> about that?
Your question has opened a whole can of worms on my side :(, but isn't this
code saying two things at once?
- it's only for calculating checksums only for BLCKSZ-sized blocks of
relations (I could imagine some user-placed file, or ENOSPC failure)
- we expect pread/pwrite() atomicity between two concurrent backends (one
backend doing pwrite()/fallocate() extends or truncation while the 2nd
is doing basebackup?)
bbsink_begin_backup(buffer_length=SINK_BUFFER_LENGTH) will set sink->
bbs_buffer_length to that SINK.. value and even there verify too:
Assert((sink->bbs_buffer_length % BLCKSZ) == 0);
sendFile() has the same (+ there's more occurences of this)
/*
* Checksums are verified in multiples of BLCKSZ, so the buffer length
* should be a multiple of the block size as well.
*/
Assert((sink->bbs_buffer_length % BLCKSZ) == 0);
further more sendFile() has this too above the "if (cnt < BLCKSZ)"
/*
* If we get a partial read, that must mean that the relation is
* being truncated. Ultimately, it should be truncated to a
* multiple of BLCKSZ, since this path should only be reached for
* relation files, but we might transiently observe an
* intermediate value.
*
* It should be fine to treat this just as if the entire block had
* been truncated away - i.e. fill this and all later blocks with
* zeroes. WAL replay will fix things up.
*/
and below that, there is ereport() that says just checksums couldn't be
verified. I mean in 0004 we are just growing and so it assumes we shouldn't
get any problems with this code. It looks like it is written while expecting
proper pread()/pwrite() atomicity when extending for up to 32kb, or am I
missing something?
On 6.17.x and 6.14.x x86_64 ext4 I've got no issues with such atomicity for
buffered preads while __appending__ stuff (using pwrite() to extend) and pread()
got correct stuff or premature EOF - that was for both 8kB and 256kB. Of
course I'm able to get torn-pages (only on 6.17.x - sic!) while doing buffered
OVERWRITES (so yea, 8kb split into 4kB + old 4KB -- checksum wouldn't match]).
Anyway as this topic touches the lack of pread()/pwrite() atomicity, but:
a) for torn-pages we are already protected there (see nearby comment for
reread_cnt , so we re-read)
b) for truncation too we are protected too
c) and it seems there is no issue for extending/appending
I've done some extensive pgbench read-write tests with pg_basebackup back then
and that such stuff didn't happen, perhaps buildfarm would tell more(?) My
only worry is that there could be some silent kernel change (in the past)
that would allow too see incompletle data by pread() for pwrite(), but if
that is not happening on 8kB/32kB (which is already over page-size), why
would that be happening for higher values?
BTW: previous such discussions [1][2][3][4] all seem to point that lack of
similar atomicity guaranteees or seem to be kernel bugs or some bizzare platform
(see e.g. "pwrite/pread is non-atomic" in [2]) or at least deficit in fs
(while XFS being always safer [5]).
Still, even if if we just read miss some stale data due to pread() woes (and
skip checksums there) the data should be recoverable from WAL.
-J.
[1] -
https://www.postgresql.org/message-id/20220116071210.GA735692%40rfd.leadboat.com
[2] -
https://www.postgresql.org/message-id/20220116210241.GC756210%40rfd.leadboat.com
[3] -
https://www.postgresql.org/message-id/CA%2BhUKGJ98%3DMOjDCnMAC5gSpkzrrey%3DO%2BaEQJ1OY03C%3DcVtiwkA%40mail.gmail.com
[4] -
https://www.postgresql.org/message-id/CA%2BhUKG%2B19bZKidSiWmMsDmgUVe%3D_rr0m57LfR%2BnAbWprVDd_cw%40mail.gmail.com
[5] -
https://www.postgresql.org/message-id/CA%2BhUKGJsT8G_YyjUzMZaJTWyua6PbwC3TAUMv_kDS0F0vzr2Pw%40mail.gmail.com
- after "As for ext4, we've detected and debugged clues.."
From 7343112175cd8e917641afa93a8a73130c7c2ea8 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Thu, 30 Jul 2026 11:02:42 +0200
Subject: [PATCH v06102026 02/10] pg_basebackup: add new "client-blackhole"
(benchmarking) backup target
Accept a new "client-blackhole" target value for pg_basebackup's --target
option. Unlike the server-side "server-blackhole" target, the whole backup
is sent to pg_basebackup as usual, but instead of being written to disk it is
discarded: regular file contents go to the /dev/null device and
directories/symbolic links/backup manifest are not created. In tar format
the archive itself is sent to the /dev/null device too.
This is a testing and development feature. It allows measuring how fast a
backup can be produced by the server and network without being limited by
the speed of the local storage (but subject to pg_basebackup
single-threaded limitations like TLS decryption and/or checksum
validation).
The client-side discard is implemented in the astreamer extractors, which
is controlled by a new backup_target_clientblackhole flag (backup_target
stays NULL).
Author: Jakub Wartak <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/CAKZiRmwwW-hDc3B6ERJB%2BpaX7RNSBcQLheq1KdsTf42cGuRvuA%40mail.gmail.com
---
doc/src/sgml/ref/pg_basebackup.sgml | 7 +-
src/bin/pg_basebackup/pg_basebackup.c | 115 +++++++++++++++----
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 18 +++
src/fe_utils/astreamer_file.c | 36 ++++--
src/include/fe_utils/astreamer.h | 3 +-
5 files changed, 150 insertions(+), 29 deletions(-)
diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml
index 6ae7637afb9..fbb249b6382 100644
--- a/doc/src/sgml/ref/pg_basebackup.sgml
+++ b/doc/src/sgml/ref/pg_basebackup.sgml
@@ -269,7 +269,12 @@ PostgreSQL documentation
server requires superuser privileges or having privileges of the
<literal>pg_write_server_files</literal> role. If the target is set to
<literal>server-blackhole</literal>, the contents are discarded by the
- server and not stored anywhere. This should only be used for testing
+ server and not stored anywhere. If the target is set to
+ <literal>client-blackhole</literal>, the backup is sent to
+ <application>pg_basebackup</application> as usual, but discarded instead
+ of being written to disk; this can be used to measure how fast a backup
+ can be produced without being limited by the speed of the local
+ storage. Both blackhole targets should only be used for testing
purposes, as you will not end up with an actual backup.
</para>
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index c3b87a19e76..4628900da10 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -156,6 +156,7 @@ static bool manifest = true;
static bool manifest_force_encode = false;
static char *manifest_checksums = NULL;
static DataDirSyncMethod sync_method = DATA_DIR_SYNC_METHOD_FSYNC;
+static bool backup_target_clientblackhole = false;
static bool success = false;
static bool made_new_pgdata = false;
@@ -1146,7 +1147,8 @@ CreateBackupStreamer(char *archive_name, char *spclocation,
directory = get_tablespace_mapping(spclocation);
streamer = astreamer_extractor_new(directory,
get_tablespace_mapping,
- progress_update_filename);
+ progress_update_filename,
+ backup_target_clientblackhole);
}
else
{
@@ -1158,8 +1160,15 @@ CreateBackupStreamer(char *archive_name, char *spclocation,
* Normally, we write it to the archive name provided by the caller,
* but when the base directory is "-" that means we need to write to
* standard output.
+ *
+ * When discarding writes, we send the archive to the null device.
*/
- if (strcmp(basedir, "-") == 0)
+ if (backup_target_clientblackhole)
+ {
+ snprintf(archive_filename, sizeof(archive_filename), "%s", DEVNULL);
+ archive_file = NULL;
+ }
+ else if (strcmp(basedir, "-") == 0)
{
snprintf(archive_filename, sizeof(archive_filename), "-");
archive_file = stdout;
@@ -1171,25 +1180,32 @@ CreateBackupStreamer(char *archive_name, char *spclocation,
archive_file = NULL;
}
+ /*
+ * Setup streamer. In case of using client blackhole, it does not make sense
+ * to append compression suffixes.
+ */
if (compress->algorithm == PG_COMPRESSION_NONE)
streamer = astreamer_plain_writer_new(archive_filename,
archive_file);
else if (compress->algorithm == PG_COMPRESSION_GZIP)
{
- strlcat(archive_filename, ".gz", sizeof(archive_filename));
+ if (!backup_target_clientblackhole)
+ strlcat(archive_filename, ".gz", sizeof(archive_filename));
streamer = astreamer_gzip_writer_new(archive_filename,
archive_file, compress);
}
else if (compress->algorithm == PG_COMPRESSION_LZ4)
{
- strlcat(archive_filename, ".lz4", sizeof(archive_filename));
+ if (!backup_target_clientblackhole)
+ strlcat(archive_filename, ".lz4", sizeof(archive_filename));
streamer = astreamer_plain_writer_new(archive_filename,
archive_file);
streamer = astreamer_lz4_compressor_new(streamer, compress);
}
else if (compress->algorithm == PG_COMPRESSION_ZSTD)
{
- strlcat(archive_filename, ".zst", sizeof(archive_filename));
+ if (!backup_target_clientblackhole)
+ strlcat(archive_filename, ".zst", sizeof(archive_filename));
streamer = astreamer_plain_writer_new(archive_filename,
archive_file);
streamer = astreamer_zstd_compressor_new(streamer, compress);
@@ -1470,6 +1486,15 @@ ReceiveArchiveStreamChunk(size_t r, char *copybuf, void *callback_data)
*/
if (state->manifest_inject_streamer != NULL)
state->manifest_buffer = createPQExpBuffer();
+ else if (backup_target_clientblackhole)
+ {
+ /* Throw away the manifest too */
+ snprintf(state->manifest_filename,
+ sizeof(state->manifest_filename), "%s", DEVNULL);
+ state->manifest_file = fopen(DEVNULL, "wb");
+ if (state->manifest_file == NULL)
+ pg_fatal("could not open file \"%s\": %m", DEVNULL);
+ }
else
{
snprintf(state->manifest_filename,
@@ -1683,11 +1708,22 @@ ReceiveBackupManifest(PGconn *conn)
{
WriteManifestState state;
- snprintf(state.filename, sizeof(state.filename),
- "%s/backup_manifest.tmp", basedir);
- state.file = fopen(state.filename, "wb");
- if (state.file == NULL)
- pg_fatal("could not create file \"%s\": %m", state.filename);
+ if (backup_target_clientblackhole)
+ {
+ /* Throw away the manifest */
+ snprintf(state.filename, sizeof(state.filename), "%s", DEVNULL);
+ state.file = fopen(DEVNULL, "wb");
+ if (state.file == NULL)
+ pg_fatal("could not open file \"%s\": %m", DEVNULL);
+ }
+ else
+ {
+ snprintf(state.filename, sizeof(state.filename),
+ "%s/backup_manifest.tmp", basedir);
+ state.file = fopen(state.filename, "wb");
+ if (state.file == NULL)
+ pg_fatal("could not create file \"%s\": %m", state.filename);
+ }
ReceiveCopyData(conn, ReceiveBackupManifestChunk, &state);
@@ -1970,6 +2006,9 @@ BaseBackup(char *compression_algorithm, char *compression_detail,
compression_detail);
}
+ if (verbose && backup_target_clientblackhole)
+ pg_log_info("the backup is being discarded and cannot be used for recovery");
+
if (verbose)
pg_log_info("initiating base backup, waiting for checkpoint to complete");
@@ -2052,7 +2091,8 @@ BaseBackup(char *compression_algorithm, char *compression_detail,
* won't be storing anything into these directories and thus should
* not create them.
*/
- if (backup_target == NULL && format == 'p' && !PQgetisnull(res, i, 1))
+ if (backup_target == NULL && !backup_target_clientblackhole && format == 'p' &&
+ !PQgetisnull(res, i, 1))
{
char *path = PQgetvalue(res, i, 1);
@@ -2276,11 +2316,8 @@ BaseBackup(char *compression_algorithm, char *compression_detail,
* synced after being completed. In plain format, all the data of the
* base directory is synced, taking into account all the tablespaces.
* Errors are not considered fatal.
- *
- * If, however, there's a backup target, we're not writing anything
- * locally, so in that case we skip this step.
*/
- if (do_sync && backup_target == NULL)
+ if (do_sync && backup_target == NULL && !backup_target_clientblackhole)
{
if (verbose)
pg_log_info("syncing data to disk ...");
@@ -2302,7 +2339,7 @@ BaseBackup(char *compression_algorithm, char *compression_detail,
* without a backup_manifest file, decreasing the chances that a directory
* we leave behind will be mistaken for a valid backup.
*/
- if (!writing_to_stdout && manifest && backup_target == NULL)
+ if (!writing_to_stdout && manifest && backup_target == NULL && !backup_target_clientblackhole)
{
char tmp_filename[MAXPGPATH];
char filename[MAXPGPATH];
@@ -2479,7 +2516,15 @@ main(int argc, char **argv)
temp_replication_slot = false;
break;
case 't':
- backup_target = pg_strdup(optarg);
+
+ /*
+ * Target: everything else than "client-blackhole" is passed
+ * through to the server as the backup target.
+ */
+ if (strcmp(optarg, "client-blackhole") == 0)
+ backup_target_clientblackhole = true;
+ else
+ backup_target = pg_strdup(optarg);
break;
case 'T':
tablespace_list_append(optarg);
@@ -2579,6 +2624,17 @@ main(int argc, char **argv)
backup_target = NULL;
}
+ if (backup_target_clientblackhole)
+ {
+ if (basedir != NULL || backup_target != NULL)
+ {
+ pg_log_error("cannot specify both output directory and backup target");
+ pg_log_error_hint("Try \"%s --help\" for more information.", progname);
+ exit(1);
+ }
+ basedir = pg_strdup(DEVNULL);
+ }
+
/*
* Can't use --format with --target. Without --target, default format is
* tar.
@@ -2671,6 +2727,23 @@ main(int argc, char **argv)
exit(1);
}
+ /*
+ * The "client-blackhole" target throws the backup away as it is received.
+ * As we do not have directories prepared, we cannot stream or write recovery
+ * configuration.
+ */
+ if (backup_target_clientblackhole)
+ {
+ if (includewal == STREAM_WAL)
+ {
+ pg_log_error("WAL cannot be streamed with the \"client-blackhole\" backup target");
+ pg_log_error_hint("Use \"%s\" or \"%s\".", "-X none", "-X fetch");
+ exit(1);
+ }
+ if (writerecoveryconf)
+ pg_fatal("recovery configuration cannot be written with the \"client-blackhole\" backup target");
+ }
+
/*
* Sanity checks for WAL method.
*/
@@ -2824,9 +2897,11 @@ main(int argc, char **argv)
/*
* If an output directory was specified, verify that it exists, or create
* it. Note that for a tar backup, an output directory of "-" means we are
- * writing to stdout, so do nothing in that case.
+ * writing to stdout, so do nothing in that case. When discarding writes,
+ * we don't create anything on disk at all.
*/
- if (basedir != NULL && (format == 'p' || strcmp(basedir, "-") != 0))
+ if (basedir != NULL && !backup_target_clientblackhole &&
+ (format == 'p' || strcmp(basedir, "-") != 0))
verify_dir_is_empty_or_create(basedir, &made_new_pgdata, &found_existing_pgdata);
/* determine remote server's xlog segment size */
@@ -2834,7 +2909,7 @@ main(int argc, char **argv)
exit(1);
/* Create pg_wal symlink, if required */
- if (xlog_dir)
+ if (xlog_dir && !backup_target_clientblackhole)
{
char *linkloc;
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 17977b474b8..33fb446ba52 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -324,6 +324,24 @@ $node->command_ok(
ok(-f "$tempdir/tarbackup/base.tar", 'backup tar was created');
rmtree("$tempdir/tarbackup");
+# The "client-blackhole" target receives the whole backup but throws it away.
+$node->command_ok(
+ [
+ @pg_basebackup_defs,
+ '--target' => 'client-blackhole',
+ '--format' => 'plain',
+ '--wal-method' => 'none'
+ ],
+ 'client-blackhole target in plain format');
+$node->command_ok(
+ [
+ @pg_basebackup_defs,
+ '--target' => 'client-blackhole',
+ '--format' => 'tar',
+ '--wal-method' => 'fetch'
+ ],
+ 'client-blackhole target in tar format');
+
$node->command_fails_like(
[
@pg_basebackup_defs,
diff --git a/src/fe_utils/astreamer_file.c b/src/fe_utils/astreamer_file.c
index fb36cecc22a..e4fa2596b08 100644
--- a/src/fe_utils/astreamer_file.c
+++ b/src/fe_utils/astreamer_file.c
@@ -37,6 +37,7 @@ typedef struct astreamer_extractor
void (*report_output_file) (const char *);
char filename[MAXPGPATH];
FILE *file;
+ bool discard_backup;
} astreamer_extractor;
static void astreamer_plain_writer_content(astreamer *streamer,
@@ -60,7 +61,8 @@ static void astreamer_extractor_finalize(astreamer *streamer);
static void astreamer_extractor_free(astreamer *streamer);
static void extract_directory(const char *filename, mode_t mode);
static void extract_link(const char *filename, const char *linktarget);
-static FILE *create_file_for_extract(const char *filename, mode_t mode);
+static FILE *create_file_for_extract(const char *filename, mode_t mode,
+ bool discard_backup);
static const astreamer_ops astreamer_extractor_ops = {
.content = astreamer_extractor_content,
@@ -181,11 +183,15 @@ astreamer_plain_writer_free(astreamer *streamer)
* 'report_output_file' is a function that will be called each time we open a
* new output file. The pathname to that file is passed as an argument. If
* NULL, the call is skipped.
+ *
+ * If 'discard_backup' is true, the extracted archive is thrown away rather
+ * than written to the filesystem.
*/
astreamer *
astreamer_extractor_new(const char *basepath,
const char *(*link_map) (const char *),
- void (*report_output_file) (const char *))
+ void (*report_output_file) (const char *),
+ bool discard_backup)
{
astreamer_extractor *streamer;
@@ -195,6 +201,7 @@ astreamer_extractor_new(const char *basepath,
streamer->basepath = pstrdup(basepath);
streamer->link_map = link_map;
streamer->report_output_file = report_output_file;
+ streamer->discard_backup = discard_backup;
return &streamer->base;
}
@@ -231,13 +238,19 @@ astreamer_extractor_content(astreamer *streamer, astreamer_member *member,
if (mystreamer->filename[fnamelen - 1] == '/')
mystreamer->filename[fnamelen - 1] = '\0';
- /* Dispatch based on file type. */
+ /*
+ * Dispatch based on file type.
+ */
if (member->is_regular)
mystreamer->file =
create_file_for_extract(mystreamer->filename,
- member->mode);
+ member->mode,
+ mystreamer->discard_backup);
else if (member->is_directory)
- extract_directory(mystreamer->filename, member->mode);
+ {
+ if (!mystreamer->discard_backup)
+ extract_directory(mystreamer->filename, member->mode);
+ }
else if (member->is_symlink)
{
const char *linktarget = member->linktarget;
@@ -252,7 +265,8 @@ astreamer_extractor_content(astreamer *streamer, astreamer_member *member,
member->linktarget);
}
- extract_link(mystreamer->filename, linktarget);
+ if (!mystreamer->discard_backup)
+ extract_link(mystreamer->filename, linktarget);
}
/* Report output file change. */
@@ -369,10 +383,18 @@ extract_link(const char *filename, const char *linktarget)
* Return the resulting handle so we can write the content to the file.
*/
static FILE *
-create_file_for_extract(const char *filename, mode_t mode)
+create_file_for_extract(const char *filename, mode_t mode, bool discard_backup)
{
FILE *file;
+ if (discard_backup)
+ {
+ file = fopen(DEVNULL, "wb");
+ if (file == NULL)
+ pg_fatal("could not open file \"%s\": %m", DEVNULL);
+ return file;
+ }
+
file = fopen(filename, "wb");
if (file == NULL)
pg_fatal("could not create file \"%s\": %m", filename);
diff --git a/src/include/fe_utils/astreamer.h b/src/include/fe_utils/astreamer.h
index 8329e4efbc5..7206dc0c48b 100644
--- a/src/include/fe_utils/astreamer.h
+++ b/src/include/fe_utils/astreamer.h
@@ -215,7 +215,8 @@ extern astreamer *astreamer_gzip_writer_new(char *pathname, FILE *file,
pg_compress_specification *compress);
extern astreamer *astreamer_extractor_new(const char *basepath,
const char *(*link_map) (const char *),
- void (*report_output_file) (const char *));
+ void (*report_output_file) (const char *),
+ bool discard_backup);
extern astreamer *astreamer_gzip_decompressor_new(astreamer *next);
extern astreamer *astreamer_lz4_compressor_new(astreamer *next,
--
2.43.0
From d294515f86fb1ed19b2bade5e790235cb2477114 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Thu, 30 Jul 2026 11:02:13 +0200
Subject: [PATCH v06102026 01/10] pg_basebackup: rename the "blackhole" backup
target to "server-blackhole"
The built-in "blackhole" backup target discards the backup on the server
side. Rename it to "server-blackhole" to make the location of the discard
explicit and to leave room for a "client-blackhole" in the follow-up
commit.
Author: Jakub Wartak <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/CAKZiRmwwW-hDc3B6ERJB%2BpaX7RNSBcQLheq1KdsTf42cGuRvuA%40mail.gmail.com
---
doc/src/sgml/protocol.sgml | 2 +-
doc/src/sgml/ref/pg_basebackup.sgml | 6 +++---
src/backend/backup/basebackup_target.c | 4 ++--
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 12 ++++++------
4 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 8ced69f283b..4995a2425b5 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -3163,7 +3163,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
sent to the client. If it is <literal>server</literal>, the backup
data is written to the server at the pathname specified by the
<literal>TARGET_DETAIL</literal> option. If it is
- <literal>blackhole</literal>, the backup data is not sent
+ <literal>server-blackhole</literal>, the backup data is not sent
anywhere; it is simply discarded.
</para>
diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml
index 3117968d125..6ae7637afb9 100644
--- a/doc/src/sgml/ref/pg_basebackup.sgml
+++ b/doc/src/sgml/ref/pg_basebackup.sgml
@@ -268,9 +268,9 @@ PostgreSQL documentation
<literal>/some/path</literal> directory. Storing a backup on the
server requires superuser privileges or having privileges of the
<literal>pg_write_server_files</literal> role. If the target is set to
- <literal>blackhole</literal>, the contents are discarded and not
- stored anywhere. This should only be used for testing purposes, as you
- will not end up with an actual backup.
+ <literal>server-blackhole</literal>, the contents are discarded by the
+ server and not stored anywhere. This should only be used for testing
+ purposes, as you will not end up with an actual backup.
</para>
<para>
diff --git a/src/backend/backup/basebackup_target.c b/src/backend/backup/basebackup_target.c
index 1c250d2895c..61465458638 100644
--- a/src/backend/backup/basebackup_target.c
+++ b/src/backend/backup/basebackup_target.c
@@ -40,7 +40,7 @@ static void *server_check_detail(char *target, char *target_detail);
static BaseBackupTargetType builtin_backup_targets[] =
{
{
- "blackhole", reject_target_detail, blackhole_get_sink
+ "server-blackhole", reject_target_detail, blackhole_get_sink
},
{
"server", server_check_detail, server_get_sink
@@ -185,7 +185,7 @@ initialize_target_list(void)
/*
* Normally, a get_sink function should construct and return a new bbsink that
- * implements the backup target, but the 'blackhole' target just throws the
+ * implements the backup target, but the 'server-blackhole' target just throws the
* data away. We could implement that by adding a bbsink that does nothing
* but forward, but it's even cheaper to implement that by not adding a bbsink
* at all.
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 2442131e179..17977b474b8 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -683,13 +683,13 @@ $node->command_ok(
'pg_basebackup --wal-method fetch runs');
$node->command_fails_like(
- [ @pg_basebackup_defs, '--target' => 'blackhole' ],
+ [ @pg_basebackup_defs, '--target' => 'server-blackhole' ],
qr/WAL cannot be streamed when a backup target is specified/,
'backup target requires --wal-method');
$node->command_fails_like(
[
@pg_basebackup_defs,
- '--target' => 'blackhole',
+ '--target' => 'server-blackhole',
'--wal-method' => 'stream'
],
qr/WAL cannot be streamed when a backup target is specified/,
@@ -701,7 +701,7 @@ $node->command_fails_like(
$node->command_fails_like(
[
@pg_basebackup_defs,
- '--target' => 'blackhole',
+ '--target' => 'server-blackhole',
'--wal-method' => 'none',
'--pgdata' => "$tempdir/blackhole"
],
@@ -710,7 +710,7 @@ $node->command_fails_like(
$node->command_fails_like(
[
@pg_basebackup_defs,
- '--target' => 'blackhole',
+ '--target' => 'server-blackhole',
'--wal-method' => 'none',
'--format' => 'tar'
],
@@ -719,7 +719,7 @@ $node->command_fails_like(
$node->command_ok(
[
@pg_basebackup_defs,
- '--target' => 'blackhole',
+ '--target' => 'server-blackhole',
'--wal-method' => 'none'
],
'backup target blackhole');
@@ -780,7 +780,7 @@ $node->command_fails_like(
$node->command_fails_like(
[
@pg_basebackup_defs,
- '--target' => 'blackhole',
+ '--target' => 'server-blackhole',
'--pgdata' => "$tempdir/blackhole"
],
qr/cannot specify both output directory and backup target/,
--
2.43.0
From ed05777dfa6f0a7ca766d613b877c3a01f4920b2 Mon Sep 17 00:00:00 2001
From: Gustavo William <[email protected]>
Date: Tue, 15 Sep 2026 13:15:51 -0400
Subject: [PATCH v06102026 05/10] basebackup: issue posix_fadvise() for more
efficient cold-cache handling
For full backups, inform the OS that we are going to need the whole
file being backed up, so it can perform more efficient read-ahead.
This in turn allows our synchronous basebackup_read_file()->pread64()
calls to see lower latencies, since the data is served from the page
cache more often, increasing the overall throughput of the transfer.
For incremental backups, issue posix_fadvise(POSIX_FADV_WILLNEED)
hints for upcoming blocks before the read loop reaches them, giving
the kernel a chance to fetch them into the page cache asynchronously
while we're still processing the current block. How far ahead we
prefetch is bounded by the maintenance_io_concurrency GUC. Contiguous
blocks are merged into a single read-ahead request to cut down on the
number of syscalls and improve I/O efficiency.
Co-authored-by: Gustavo William <[email protected]>
Co-authored-by: Jakub Wartak <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/CAKZiRmwwW-hDc3B6ERJB%2BpaX7RNSBcQLheq1KdsTf42cGuRvuA%40mail.gmail.com
---
src/backend/backup/basebackup.c | 94 ++++++++++++++++++++++++++++++++-
1 file changed, 93 insertions(+), 1 deletion(-)
diff --git a/src/backend/backup/basebackup.c b/src/backend/backup/basebackup.c
index 6478218cc91..e1039569c82 100644
--- a/src/backend/backup/basebackup.c
+++ b/src/backend/backup/basebackup.c
@@ -12,6 +12,7 @@
*/
#include "postgres.h"
+#include <fcntl.h>
#include <sys/stat.h>
#include <unistd.h>
#include <time.h>
@@ -38,6 +39,7 @@
#include "replication/slot.h"
#include "replication/walsender.h"
#include "replication/walsender_private.h"
+#include "storage/bufmgr.h"
#include "storage/bufpage.h"
#include "storage/checksum.h"
#include "storage/dsm_impl.h"
@@ -106,6 +108,13 @@ static off_t read_file_data_into_buffer(bbsink *sink,
BlockNumber blkno,
bool verify_checksum,
int *checksum_failures);
+#if defined(USE_POSIX_FADVISE) && defined(POSIX_FADV_WILLNEED)
+static void prefetch_next_incremental_run(int fd,
+ BlockNumber *incremental_blocks,
+ unsigned num_incremental_blocks,
+ unsigned max_run_len,
+ unsigned *prefetch_idx);
+#endif
static void push_to_sink(bbsink *sink, pg_checksum_context *checksum_ctx,
size_t *bytes_done, void *data, size_t length);
static bool backup_checksums_verifiable(XLogRecPtr start_lsn);
@@ -1591,7 +1600,11 @@ sendFile(bbsink *sink, const char *readfilename, const char *tarfilename,
pgoff_t bytes_done = 0;
bool verify_checksum = false;
pg_checksum_context checksum_ctx;
- int ibindex = 0;
+ unsigned ibindex = 0;
+#if defined(USE_POSIX_FADVISE) && defined(POSIX_FADV_WILLNEED)
+ unsigned prefetch_idx = 0;
+ unsigned prefetch_window_size = maintenance_io_concurrency;
+#endif
if (pg_checksum_init(&checksum_ctx, manifest->checksum_type) < 0)
elog(ERROR, "could not initialize checksum of file \"%s\"",
@@ -1607,6 +1620,36 @@ sendFile(bbsink *sink, const char *readfilename, const char *tarfilename,
errmsg("could not open file \"%s\": %m", readfilename)));
}
+ /*
+ * On full backups, let the OS know that we are going to read the whole file.
+ * It's just an hint, but it helps avoid longer synchronous read stalls.
+ */
+#if defined(USE_POSIX_FADVISE) && defined(POSIX_FADV_SEQUENTIAL)
+ if (incremental_blocks == NULL)
+ (void) posix_fadvise(fd, 0, 0, POSIX_FADV_SEQUENTIAL);
+#endif
+
+ /*
+ * On incremental backups, let the OS know the exact blocks we'll need.
+ * For that, maintain a bounded window of readahead hints, sized by
+ * maintenance_io_concurrency: we keep at least N runs fadvised ahead
+ * of what we're currently reading (N = maintenance_io_concurrency),
+ * which increases I/O concurrency.
+ *
+ * Initially, advance the window by triggering prefetch_window_size runs.
+ */
+#if defined(USE_POSIX_FADVISE) && defined(POSIX_FADV_WILLNEED)
+ if (incremental_blocks != NULL && prefetch_window_size > 1)
+ {
+ int initial_prefetch = prefetch_window_size;
+
+ while (initial_prefetch-- > 0 && prefetch_idx < num_incremental_blocks)
+ prefetch_next_incremental_run(fd, incremental_blocks,
+ num_incremental_blocks,
+ prefetch_window_size, &prefetch_idx);
+ }
+#endif
+
_tarWriteHeader(sink, tarfilename, NULL, statbuf, false);
/*
@@ -1730,6 +1773,7 @@ sendFile(bbsink *sink, const char *readfilename, const char *tarfilename,
* supposed to include.
*/
relative_blkno = incremental_blocks[ibindex++];
+
cnt = read_file_data_into_buffer(sink, readfilename, fd,
relative_blkno * BLCKSZ,
BLCKSZ,
@@ -1737,6 +1781,15 @@ sendFile(bbsink *sink, const char *readfilename, const char *tarfilename,
verify_checksum,
&checksum_failures);
+#if defined(USE_POSIX_FADVISE) && defined(POSIX_FADV_WILLNEED)
+ /* Advance the prefetch window by prefetching another run */
+ if (prefetch_window_size > 1 && prefetch_idx < num_incremental_blocks)
+ prefetch_next_incremental_run(fd, incremental_blocks,
+ num_incremental_blocks,
+ prefetch_window_size,
+ &prefetch_idx);
+#endif
+
/*
* If we get a partial read, that must mean that the relation is
* being truncated. Ultimately, it should be truncated to a
@@ -1840,6 +1893,45 @@ sendFile(bbsink *sink, const char *readfilename, const char *tarfilename,
return true;
}
+#if defined(USE_POSIX_FADVISE) && defined(POSIX_FADV_WILLNEED)
+
+/*
+ * Prefetch the next run of blocks of a file that will be needed soon.
+ *
+ * Advance *prefetch_idx past the next run of block numbers in incremental_blocks
+ * and issue a single readahead hint covering that whole run. This is advisory only:
+ * posix_fadvise() failures are ignored, since the worst that happens is that we
+ * don't get the intended prefetching benefit.
+ *
+ * Note that a "run" does not mean a "block": contiguous blocks are merged into
+ * the same run (up to max_run_len blocks) and fadvised altogether.
+ */
+static void
+prefetch_next_incremental_run(int fd, BlockNumber *incremental_blocks,
+ unsigned num_incremental_blocks,
+ unsigned max_run_len,
+ unsigned *prefetch_idx)
+{
+ BlockNumber run_start = incremental_blocks[(*prefetch_idx)++];
+ unsigned run_len = 1;
+
+ /*
+ * Merge contiguous blocks into a single run, up to max_run_len
+ */
+ while (run_len < max_run_len &&
+ *prefetch_idx < num_incremental_blocks &&
+ incremental_blocks[*prefetch_idx] == run_start + run_len)
+ {
+ run_len++;
+ (*prefetch_idx)++;
+ }
+
+ (void) posix_fadvise(fd, (off_t) run_start * BLCKSZ,
+ (off_t) run_len * BLCKSZ,
+ POSIX_FADV_WILLNEED);
+}
+#endif
+
/*
* Read some more data from the file into the bbsink's buffer, verifying
* checksums as required.
--
2.43.0
From 9c21143f800d9360f513a2f85387ada54bd4c3d2 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 24 Jul 2026 08:30:20 +0200
Subject: [PATCH v06102026 03/10] pg_basebackup: report average data transfer
throughput in verbose mode
Add the average data transfer rate to the "base backup completed" message.
The transfer is timed with instr_time from just before the archive data is
received until all data has arrived (but before fsync is called).
Author: Jakub Wartak <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/CAKZiRmwwW-hDc3B6ERJB%2BpaX7RNSBcQLheq1KdsTf42cGuRvuA%40mail.gmail.com
---
src/bin/pg_basebackup/pg_basebackup.c | 31 ++++++++++++++++++++++++++-
1 file changed, 30 insertions(+), 1 deletion(-)
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index 4628900da10..b6e469d47bb 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -37,6 +37,7 @@
#include "fe_utils/recovery_gen.h"
#include "getopt_long.h"
#include "libpq/protocol.h"
+#include "portability/instr_time.h"
#include "receivelog.h"
#include "streamutil.h"
@@ -1790,6 +1791,8 @@ BaseBackup(char *compression_algorithm, char *compression_detail,
int writing_to_stdout;
bool use_new_option_syntax = false;
PQExpBufferData buf;
+ instr_time transfer_start;
+ instr_time transfer_elapsed;
Assert(conn != NULL);
initPQExpBuffer(&buf);
@@ -2145,6 +2148,9 @@ BaseBackup(char *compression_algorithm, char *compression_detail,
wal_compress_level);
}
+ /* Start timing the data transfer, for the average rate report. */
+ INSTR_TIME_SET_CURRENT(transfer_start);
+
if (serverMajor >= 1500)
{
/* Receive a single tar stream with everything. */
@@ -2197,6 +2203,13 @@ BaseBackup(char *compression_algorithm, char *compression_detail,
ReceiveBackupManifest(conn);
}
+ /*
+ * All backup data has been received, so measure how long the transfer
+ * took for the average rate report shown at completion.
+ */
+ INSTR_TIME_SET_CURRENT(transfer_elapsed);
+ INSTR_TIME_SUBTRACT(transfer_elapsed, transfer_start);
+
if (showprogress)
{
progress_update_filename(NULL);
@@ -2365,7 +2378,22 @@ BaseBackup(char *compression_algorithm, char *compression_detail,
}
if (verbose)
- pg_log_info("base backup completed");
+ {
+ double elapsed_sec = INSTR_TIME_GET_DOUBLE(transfer_elapsed);
+
+ /*
+ * Avoids potential division by zero.
+ *
+ * Timing does not include potential fsync()/syncfs(), so data might be
+ * still in-flight from pagecache when total_done was calculated, therefore
+ * we make it clear to the user what we are measuring.
+ */
+ if (elapsed_sec > 0.0)
+ pg_log_info("base backup completed (avg %.1f MB/s)",
+ (double) totaldone / (1024 * 1024) / elapsed_sec);
+ else
+ pg_log_info("base backup completed");
+ }
}
@@ -2422,6 +2450,7 @@ main(int argc, char **argv)
pg_logging_init(argv[0]);
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_basebackup"));
+ pg_initialize_timing();
if (argc > 1)
{
--
2.43.0
From 4d46a93398ae4da3b73d9377234bafb5c7ae45eb Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 31 Jul 2026 10:13:57 +0200
Subject: [PATCH v06102026 04/10] basebackup: bump SINK_BUFFER_LENGTH to 256kB
By increasing SINK_BUFFER_LENGTH size to be closer to modern L2 CPU cache
sizes, we are also reducing frequency of the pread() system call which
also helps avoids performance penalties (and is visible as one of the
top functions when sending backups). In isolated performance runs where
pg_basebackup client did not have write it's backup to disk, this gives
~180% of the baseline (3.1GB/s -> 5.5GB/s).
Author: Jakub Wartak <[email protected]>
Reviewed-by: Heikki Linnakangas <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmwwW-hDc3B6ERJB%2BpaX7RNSBcQLheq1KdsTf42cGuRvuA%40mail.gmail.com
---
src/backend/backup/basebackup.c | 2 +-
src/backend/backup/basebackup_progress.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/backend/backup/basebackup.c b/src/backend/backup/basebackup.c
index e3c04ecd810..6478218cc91 100644
--- a/src/backend/backup/basebackup.c
+++ b/src/backend/backup/basebackup.c
@@ -59,7 +59,7 @@
* NB: The buffer size is required to be a multiple of the system block
* size, so use that value instead if it's bigger than our preference.
*/
-#define SINK_BUFFER_LENGTH Max(32768, BLCKSZ)
+#define SINK_BUFFER_LENGTH Max(256 * 1024, BLCKSZ)
typedef struct
{
diff --git a/src/backend/backup/basebackup_progress.c b/src/backend/backup/basebackup_progress.c
index f74459181d0..1d8b24dfdc9 100644
--- a/src/backend/backup/basebackup_progress.c
+++ b/src/backend/backup/basebackup_progress.c
@@ -63,7 +63,7 @@ bbsink_progress_new(bbsink *next, bool estimate_backup_size, bool incremental)
Assert(next != NULL);
- sink = palloc0_object(bbsink);
+ sink = palloc_aligned(sizeof(bbsink), PG_IO_ALIGN_SIZE, MCXT_ALLOC_ZERO);
*((const bbsink_ops **) &sink->bbs_ops) = &bbsink_progress_ops;
sink->bbs_next = next;
--
2.43.0
From 28fb007b7b3aaac5ba9038d97c60e059684059de Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 31 Jul 2026 13:51:21 +0200
Subject: [PATCH v06102026 07/10] pg_basebackup: preallocate extracted files
with posix_fallocate()
When extracting a basebackup, each output (segment/data file) grows each
one write() at a time. On filesystems with delayed allocation (e.g. ext4/XFS)
this makes every write() to find space, which shows up as top bottleneck of
the single-threaded/CPU-bount pg_basebackups's time. The tar member header
already gives us the final file size up front, so preallocate the whole file
in one go.
Author: Jakub Wartak <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/CAKZiRmwwW-hDc3B6ERJB%2BpaX7RNSBcQLheq1KdsTf42cGuRvuA%40mail.gmail.com
---
src/fe_utils/astreamer_file.c | 29 ++++++++++++++++++++++++++---
1 file changed, 26 insertions(+), 3 deletions(-)
diff --git a/src/fe_utils/astreamer_file.c b/src/fe_utils/astreamer_file.c
index b867e9489cd..32df6b09fb9 100644
--- a/src/fe_utils/astreamer_file.c
+++ b/src/fe_utils/astreamer_file.c
@@ -63,7 +63,7 @@ static void astreamer_extractor_free(astreamer *streamer);
static void extract_directory(const char *filename, mode_t mode);
static void extract_link(const char *filename, const char *linktarget);
static int create_file_for_extract(const char *filename, mode_t mode,
- bool discard_backup);
+ bool discard_backup, pgoff_t size);
static void write_file_range(int fd, const char *filename,
const char *data, int len);
@@ -241,7 +241,8 @@ astreamer_extractor_content(astreamer *streamer, astreamer_member *member,
mystreamer->fd =
create_file_for_extract(mystreamer->filename,
member->mode,
- mystreamer->discard_backup);
+ mystreamer->discard_backup,
+ member->size);
else if (member->is_directory)
{
if (!mystreamer->discard_backup)
@@ -377,7 +378,7 @@ extract_link(const char *filename, const char *linktarget)
*/
static int
create_file_for_extract(const char *filename, mode_t mode,
- bool discard_backup)
+ bool discard_backup, pgoff_t size)
{
int fd;
@@ -400,6 +401,28 @@ create_file_for_extract(const char *filename, mode_t mode,
filename);
#endif
+ /*
+ * Preallocate the file to its final size. We know the size up front
+ * from the tar member header, so this lets the filesystem allocate all
+ * the blocks in one go rather than growing the file on every write.
+ */
+#ifdef HAVE_POSIX_FALLOCATE
+ if (size > 0)
+ {
+ int rc = posix_fallocate(fd, 0, size);
+
+ /*
+ * This is just an optimization, so we ignore failures such as
+ * EINVAL/EOPNOTSUPP, however we need to properly fail on ENOSPC.
+ */
+ if (rc == ENOSPC)
+ {
+ errno = rc;
+ pg_fatal("could not preallocate file \"%s\": %m", filename);
+ }
+ }
+#endif
+
return fd;
}
--
2.43.0
From 47dd7b1d1b2040c557953c14553924861a81e60b Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Tue, 4 Aug 2026 12:00:23 +0200
Subject: [PATCH v06102026 08/10] libpq / pg_basebackup: add
PQgetCopyDataInternalBuf() for heavy transfers
At high data transfer rates(>10Gbit/s), the primary bottleneck in pg_basebackup
becomes the 2-nd memory copy done in the userspace by libpq itself (the first
one is the kernel itself copying the data from network to the userspace
socket).
Add optimized variant of PQgetCopyData(): PQgetCopyDataInternalBuf() which
avoids repeated memory allocation and copying. The other major differences are:
- we do not null terminate the string (as that is not used anyway)
- the data is only valid untill the next call (so by that time it has to
be consumed, but we can use internal libpq buffer until we receive
more)
Author: Jakub Wartak <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/CAKZiRmwwW-hDc3B6ERJB%2BpaX7RNSBcQLheq1KdsTf42cGuRvuA%40mail.gmail.com
---
src/bin/pg_basebackup/pg_basebackup.c | 10 +++--
src/interfaces/libpq/exports.txt | 1 +
src/interfaces/libpq/fe-exec.c | 26 ++++++++++++
src/interfaces/libpq/fe-protocol3.c | 58 +++++++++++++++++++++++++++
src/interfaces/libpq/libpq-fe.h | 1 +
src/interfaces/libpq/libpq-int.h | 1 +
6 files changed, 94 insertions(+), 3 deletions(-)
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index b6e469d47bb..dd09d7fd12c 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -1029,7 +1029,13 @@ ReceiveCopyData(PGconn *conn, WriteDataCallback callback,
int r;
char *copybuf;
- r = PQgetCopyData(conn, ©buf, 0);
+ /*
+ * Use the no-copy optimization: copybuf points directly into libpq's
+ * receive buffer and stays valid until the next call, which is all the
+ * callback needs (this avoids memory copy which hurts at high transfer
+ * rates).
+ */
+ r = PQgetCopyDataInternalBuf(conn, ©buf, 0);
if (r == -1)
{
/* End of chunk. */
@@ -1043,8 +1049,6 @@ ReceiveCopyData(PGconn *conn, WriteDataCallback callback,
pg_fatal("background process terminated unexpectedly");
(*callback) (r, copybuf, callback_data);
-
- PQfreemem(copybuf);
}
}
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 1e3d5bd5867..e02b92b285e 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -211,3 +211,4 @@ PQdefaultAuthDataHook 208
PQfullProtocolVersion 209
appendPQExpBufferVA 210
PQgetThreadLock 211
+PQgetCopyDataInternalBuf 212
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index 294690648bd..3ca5209db64 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -2844,6 +2844,32 @@ PQgetCopyData(PGconn *conn, char **buffer, int async)
return pqGetCopyData3(conn, buffer, async);
}
+/*
+ * PQgetCopyDataInternalBuf - variant of PQgetCopyData which bypasses memcpy()
+ * for high data transfer rates.
+ *
+ * Set *buffer to point directly into the internal receive buffer rather
+ * allocate and copy memory on every call. The *buffer is valid only until the
+ * next PQgetCopyDataInternalBuf/PQgetCopyData call on this connection, and must
+ * not be passed to PQfreemem(). The returned payload is NOT null-terminated.
+ *
+ * Otherwise it is pretty much the same as the original PQgetCopyData.
+ */
+int
+PQgetCopyDataInternalBuf(PGconn *conn, char **buffer, int async)
+{
+ *buffer = NULL; /* for all failure cases */
+ if (!conn)
+ return -2;
+ if (conn->asyncStatus != PGASYNC_COPY_OUT &&
+ conn->asyncStatus != PGASYNC_COPY_BOTH)
+ {
+ libpq_append_conn_error(conn, "no COPY in progress");
+ return -2;
+ }
+ return pqGetCopyDataInternalBuf3(conn, buffer, async);
+}
+
/*
* PQgetline - gets a newline-terminated string from the backend.
*
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 1fd427cb9be..4a2cb2d67d7 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -1970,6 +1970,64 @@ pqGetCopyData3(PGconn *conn, char **buffer, int async)
}
}
+/*
+ * pqGetCopyDataInternalBuf3 - like pqGetCopyData3, but avoids malloc and
+ * memory copying.
+ *
+ * Instead of allocating fresh buffer and copying the CopyData payload into
+ * that new memory, this sets *buffer directly into conn->inBuffer and returns
+ * its length. The message is marked consumed, and the returned pointer stays
+ * valid only until the next libpq call that reads from the socket (e.g. using
+ * pgReadData() or this call). Called must be done with the data processing
+ * before calling this again.
+ *
+ * Differences between thnis and the orginal pgGetCopyData3() are:
+ * - the caller must not free *buffer
+ * - the payload is not null-terminated
+ *
+ * The main advantage of this call is that it avoids memory copy for for
+ * high-throughput COPY.
+ */
+int
+pqGetCopyDataInternalBuf3(PGconn *conn, char **buffer, int async)
+{
+ int msgLength;
+
+ for (;;)
+ {
+ /* Collect the next input message; see pqGetCopyData3 for details. */
+ msgLength = getCopyDataMessage(conn);
+ if (msgLength < 0)
+ return msgLength; /* end-of-copy or error */
+ if (msgLength == 0)
+ {
+ /* Don't block if async read requested */
+ if (async)
+ return 0;
+ /* Need to load more data */
+ if (pqWait(true, false, conn) ||
+ pqReadData(conn) < 0)
+ return -2;
+ continue;
+ }
+
+ msgLength -= 4;
+ if (msgLength > 0)
+ {
+ /* Just use libpq internal input buffer */
+ *buffer = &conn->inBuffer[conn->inCursor];
+
+ /* Mark message consumed */
+ pqParseDone(conn, conn->inCursor + msgLength);
+
+ return msgLength;
+ }
+
+ /* Empty, so drop it and loop around for another */
+ pqParseDone(conn, conn->inCursor);
+ }
+}
+
/*
* PQgetline - gets a newline-terminated string from the backend.
*
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index f51fd620b0a..fd174b8a42f 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -564,6 +564,7 @@ extern PGnotify *PQnotifies(PGconn *conn);
extern int PQputCopyData(PGconn *conn, const char *buffer, int nbytes);
extern int PQputCopyEnd(PGconn *conn, const char *errormsg);
extern int PQgetCopyData(PGconn *conn, char **buffer, int async);
+extern int PQgetCopyDataInternalBuf(PGconn *conn, char **buffer, int async);
/* Deprecated routines for copy in/out */
extern int PQgetline(PGconn *conn, char *buffer, int length);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 933ba0d99d5..d046e89d184 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -776,6 +776,7 @@ extern void pqBuildErrorMessage3(PQExpBuffer msg, const PGresult *res,
PGVerbosity verbosity, PGContextVisibility show_context);
extern int pqGetNegotiateProtocolVersion3(PGconn *conn);
extern int pqGetCopyData3(PGconn *conn, char **buffer, int async);
+extern int pqGetCopyDataInternalBuf3(PGconn *conn, char **buffer, int async);
extern int pqGetline3(PGconn *conn, char *s, int maxlen);
extern int pqGetlineAsync3(PGconn *conn, char *buffer, int bufsize);
extern int pqEndcopy3(PGconn *conn);
--
2.43.0
From e70631abbccefdbefe237005d1e75057934d566a Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Wed, 5 Aug 2026 08:53:31 +0200
Subject: [PATCH v06102026 09/10] pg_basebackup: add support for Direct I/O and
Async I/O using liburing
Profiling pg_basebackup when doing high-data transfer reveals that it is
highly CPU-bound on kernel side inside write() due to kernel's side
page-cache memory allocations/cgroups memory accounting and copying the data
from userspace back to kernel side, just to be copied again to the device.
To avoid that we can do Direct Memory Transfers from our userspace buffers
directly to the storage device, however we cannot do use synchronous API
for that due to high individual device latency. On Linux however, we are
already using liburing for the backend code, so this commit teaches
pg_basebackup/astreamers (for writing plain and tar formats) to write using
AIO+DIO with deep I/O queues. It realistic benchmarks and with adequate
sequential storage bandwidth and no SSL encryption this makes possible for
pg_basebackup -Fp to go from ~1610MB/s to ~2975MB/s as long as the data is
hot in page-cache of the source server and the network and single stream TCP
socket performance is able to keep up.
This commits also exposes undocumented/debugging PG_BASEBACKUP_NODIO env
variable, that when set allows disabling this optimization.
Author: Jakub Wartak <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/CAKZiRmwwW-hDc3B6ERJB%2BpaX7RNSBcQLheq1KdsTf42cGuRvuA%40mail.gmail.com
---
src/bin/pg_basebackup/Makefile | 2 +-
src/bin/pg_basebackup/meson.build | 2 +-
src/bin/pg_basebackup/pg_basebackup.c | 9 +-
src/fe_utils/astreamer_file.c | 465 +++++++++++++++++++++++++-
src/fe_utils/meson.build | 2 +-
src/include/fe_utils/astreamer.h | 5 +-
6 files changed, 463 insertions(+), 22 deletions(-)
diff --git a/src/bin/pg_basebackup/Makefile b/src/bin/pg_basebackup/Makefile
index df94fc27d02..448abb71e99 100644
--- a/src/bin/pg_basebackup/Makefile
+++ b/src/bin/pg_basebackup/Makefile
@@ -42,7 +42,7 @@ BBOBJS = \
all: pg_basebackup pg_createsubscriber pg_receivewal pg_recvlogical
pg_basebackup: $(BBOBJS) $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
- $(CC) $(CFLAGS) $(BBOBJS) $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+ $(CC) $(CFLAGS) $(BBOBJS) $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) $(LIBURING_LIBS) -o $@$(X)
pg_createsubscriber: pg_createsubscriber.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
diff --git a/src/bin/pg_basebackup/meson.build b/src/bin/pg_basebackup/meson.build
index d70ce5786a2..e4cb673e933 100644
--- a/src/bin/pg_basebackup/meson.build
+++ b/src/bin/pg_basebackup/meson.build
@@ -7,7 +7,7 @@ common_sources = files(
'walmethods.c',
)
-pg_basebackup_deps = [frontend_code, libpq, lz4, zlib, zstd]
+pg_basebackup_deps = [frontend_code, libpq, lz4, zlib, zstd, liburing]
pg_basebackup_common = static_library('libpg_basebackup_common',
common_sources,
dependencies: pg_basebackup_deps,
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index dd09d7fd12c..0b571c54ab0 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -1153,7 +1153,8 @@ CreateBackupStreamer(char *archive_name, char *spclocation,
streamer = astreamer_extractor_new(directory,
get_tablespace_mapping,
progress_update_filename,
- backup_target_clientblackhole);
+ backup_target_clientblackhole,
+ verbose);
}
else
{
@@ -1191,7 +1192,7 @@ CreateBackupStreamer(char *archive_name, char *spclocation,
*/
if (compress->algorithm == PG_COMPRESSION_NONE)
streamer = astreamer_plain_writer_new(archive_filename,
- archive_file);
+ archive_file, verbose);
else if (compress->algorithm == PG_COMPRESSION_GZIP)
{
if (!backup_target_clientblackhole)
@@ -1204,7 +1205,7 @@ CreateBackupStreamer(char *archive_name, char *spclocation,
if (!backup_target_clientblackhole)
strlcat(archive_filename, ".lz4", sizeof(archive_filename));
streamer = astreamer_plain_writer_new(archive_filename,
- archive_file);
+ archive_file, verbose);
streamer = astreamer_lz4_compressor_new(streamer, compress);
}
else if (compress->algorithm == PG_COMPRESSION_ZSTD)
@@ -1212,7 +1213,7 @@ CreateBackupStreamer(char *archive_name, char *spclocation,
if (!backup_target_clientblackhole)
strlcat(archive_filename, ".zst", sizeof(archive_filename));
streamer = astreamer_plain_writer_new(archive_filename,
- archive_file);
+ archive_file, verbose);
streamer = astreamer_zstd_compressor_new(streamer, compress);
}
else
diff --git a/src/fe_utils/astreamer_file.c b/src/fe_utils/astreamer_file.c
index 32df6b09fb9..d4d34e5d412 100644
--- a/src/fe_utils/astreamer_file.c
+++ b/src/fe_utils/astreamer_file.c
@@ -17,17 +17,61 @@
#include <fcntl.h>
#include <unistd.h>
+#ifdef USE_LIBURING
+#include <liburing.h>
+#endif
#include "common/file_perm.h"
#include "common/logging.h"
#include "fe_utils/astreamer.h"
+#ifdef USE_LIBURING
+/*
+ * Parameters for the io_uring + O_DIRECT write path. Aim is to keep deep
+ * queue of the I/O devices populated by throwing large number of independent
+ * synchronous/DIRECT_IO requests, but in asynchronous way. We avoid latency
+ * of single synchronous write, and utilize the device up to the max what it
+ * allows.
+ */
+#define DIO_ALIGN 4096 /* O_DIRECT aligment */
+#define DIO_BUFSZ (1024 * 1024) /* size of each direct I/O write */
+#define DIO_NBUF 32 /* queue depth, XXX:expose it via getopt? */
+#define DIO_MIN_SIZE DIO_BUFSZ /* fsize threshold for activating O_DIRECT writes */
+#define DIO_SUBMIT_BATCH 8 /* how many SQEs to batch */
+
+/*
+ * State for the dio/io_uring writer. Used by plain(file) and tar extractors.
+ * We buffer data as we recieve it, into the "pool" of buffers and submit them
+ * using DIO_BUFSZ sizes.
+ */
+typedef struct dio_writer
+{
+ struct io_uring ring; /* see io_uring(7) */
+ bool ring_ready; /* was the ring and buffer initialized */
+ char *pool; /* buffer memory, max size: DIO_NBUF * DIO_BUFSZ */
+ bool busy[DIO_NBUF]; /* is buffer in flight? */
+ size_t buflen[DIO_NBUF]; /* bytes submitted for in-flight buffer */
+ int fd; /* DIO fd: if fd > 0 then it's active, -1 otherwise */
+ const char *filename; /* for error handling */
+ pgoff_t offset; /* offset of the next write */
+ pgoff_t written; /* bytes written */
+ int curidx; /* buffer being filled, or -1 */
+ int curlen; /* bytes filled in current buffer */
+ int inflight; /* prepared writes not yet reaped */
+ int unsubmitted; /* SQEs prepared but not yet submitted */
+ bool notified; /* already logged a fall-back-to-buffered? */
+} dio_writer;
+#endif
+
typedef struct astreamer_plain_writer
{
astreamer base;
char *pathname;
- FILE *file;
+ FILE *file; /* if NULL, then dio.fd is used */
bool should_close_file;
+#ifdef USE_LIBURING
+ dio_writer dio;
+#endif
} astreamer_plain_writer;
typedef struct astreamer_extractor
@@ -37,8 +81,12 @@ typedef struct astreamer_extractor
const char *(*link_map) (const char *);
void (*report_output_file) (const char *);
char filename[MAXPGPATH];
- int fd;
+ int fd; /* if -1, then dio.fd is used */
bool discard_backup;
+ bool verbose;
+#ifdef USE_LIBURING
+ dio_writer dio;
+#endif
} astreamer_extractor;
static void astreamer_plain_writer_content(astreamer *streamer,
@@ -66,6 +114,13 @@ static int create_file_for_extract(const char *filename, mode_t mode,
bool discard_backup, pgoff_t size);
static void write_file_range(int fd, const char *filename,
const char *data, int len);
+#ifdef USE_LIBURING
+static bool dio_writer_start(dio_writer *dw, const char *filename,
+ pgoff_t prealloc_size, bool verbose);
+static void dio_writer_write(dio_writer *dw, const char *data, int len);
+static void dio_writer_finish(dio_writer *dw);
+static void dio_writer_destroy(dio_writer *dw);
+#endif
static const astreamer_ops astreamer_extractor_ops = {
.content = astreamer_extractor_content,
@@ -83,7 +138,7 @@ static const astreamer_ops astreamer_extractor_ops = {
* there.
*/
astreamer *
-astreamer_plain_writer_new(char *pathname, FILE *file)
+astreamer_plain_writer_new(char *pathname, FILE *file, bool verbose)
{
astreamer_plain_writer *streamer;
@@ -93,13 +148,33 @@ astreamer_plain_writer_new(char *pathname, FILE *file)
streamer->pathname = pstrdup(pathname);
streamer->file = file;
+#ifdef USE_LIBURING
+ streamer->dio.fd = -1;
+#endif
if (file == NULL)
{
- streamer->file = fopen(pathname, "wb");
- if (streamer->file == NULL)
- pg_fatal("could not create file \"%s\": %m", pathname);
- streamer->should_close_file = true;
+#ifdef USE_LIBURING
+
+ /*
+ * Fallback to classic buffered writes in case of:
+ * - env variable PG_BASEBACKUP_NODIO is set (debugging)
+ * - pipe/stdout is used
+ * - /dev/null is being used (-t client-blackhole)
+ * - filesystems without O_DIRECT support
+ */
+ if (getenv("PG_BASEBACKUP_NODIO") == NULL &&
+ dio_writer_start(&streamer->dio, streamer->pathname, 0, verbose) == true) {
+ /* do not use buffered writes, as the dio.fd is going to be used */
+ streamer->file = NULL;
+ } else
+#endif
+ {
+ streamer->file = fopen(pathname, "wb");
+ if (streamer->file == NULL)
+ pg_fatal("could not create file \"%s\": %m", pathname);
+ streamer->should_close_file = true;
+ }
}
return &streamer->base;
@@ -120,6 +195,13 @@ astreamer_plain_writer_content(astreamer *streamer,
if (len == 0)
return;
+#ifdef USE_LIBURING
+ if (mystreamer->dio.fd != -1)
+ {
+ dio_writer_write(&mystreamer->dio, data, len);
+ return;
+ }
+#endif
write_file_range(fileno(mystreamer->file), mystreamer->pathname, data, len);
}
@@ -134,6 +216,13 @@ astreamer_plain_writer_finalize(astreamer *streamer)
mystreamer = (astreamer_plain_writer *) streamer;
+#ifdef USE_LIBURING
+ if (mystreamer->dio.fd != -1)
+ {
+ dio_writer_finish(&mystreamer->dio);
+ return;
+ }
+#endif
if (mystreamer->should_close_file && fclose(mystreamer->file) != 0)
pg_fatal("could not close file \"%s\": %m",
mystreamer->pathname);
@@ -156,6 +245,9 @@ astreamer_plain_writer_free(astreamer *streamer)
Assert(mystreamer->base.bbs_next == NULL);
pfree(mystreamer->pathname);
+#ifdef USE_LIBURING
+ dio_writer_destroy(&mystreamer->dio);
+#endif
pfree(mystreamer);
}
@@ -186,7 +278,7 @@ astreamer *
astreamer_extractor_new(const char *basepath,
const char *(*link_map) (const char *),
void (*report_output_file) (const char *),
- bool discard_backup)
+ bool discard_backup, bool verbose)
{
astreamer_extractor *streamer;
@@ -197,7 +289,11 @@ astreamer_extractor_new(const char *basepath,
streamer->link_map = link_map;
streamer->report_output_file = report_output_file;
streamer->discard_backup = discard_backup;
+ streamer->verbose = verbose;
streamer->fd = -1;
+#ifdef USE_LIBURING
+ streamer->dio.fd = -1;
+#endif
return &streamer->base;
}
@@ -220,6 +316,9 @@ astreamer_extractor_content(astreamer *streamer, astreamer_member *member,
{
case ASTREAMER_MEMBER_HEADER:
Assert(mystreamer->fd == -1);
+#ifdef USE_LIBURING
+ Assert(mystreamer->dio.fd == -1);
+#endif
if (!path_is_safe_for_extraction(member->pathname))
pg_fatal("tar member has unsafe path name: \"%s\"",
@@ -238,11 +337,31 @@ astreamer_extractor_content(astreamer *streamer, astreamer_member *member,
* Dispatch based on file type.
*/
if (member->is_regular)
- mystreamer->fd =
- create_file_for_extract(mystreamer->filename,
- member->mode,
- mystreamer->discard_backup,
- member->size);
+ {
+#ifdef USE_LIBURING
+ /*
+ * Fallback to classic buffered writes in case of:
+ * - small files
+ * - env variable PG_BASEBACKUP_NODIO is set (debugging)
+ * - pipe/stdout is used
+ * - /dev/null is being used (-t client-blackhole)
+ * - filesystems without O_DIRECT support
+ */
+ if (member->size >= DIO_MIN_SIZE &&
+ !mystreamer->discard_backup &&
+ getenv("PG_BASEBACKUP_NODIO") == NULL &&
+ dio_writer_start(&mystreamer->dio, mystreamer->filename,
+ member->size, mystreamer->verbose)) {
+ /* do not use buffered writes, as the dio.fd is going to be used */
+ mystreamer->fd = -1;
+ } else
+#endif
+ mystreamer->fd =
+ create_file_for_extract(mystreamer->filename,
+ member->mode,
+ mystreamer->discard_backup,
+ member->size);
+ }
else if (member->is_directory)
{
if (!mystreamer->discard_backup)
@@ -272,6 +391,14 @@ astreamer_extractor_content(astreamer *streamer, astreamer_member *member,
break;
case ASTREAMER_MEMBER_CONTENTS:
+#ifdef USE_LIBURING
+ if (mystreamer->dio.fd != -1)
+ {
+ if (len > 0)
+ dio_writer_write(&mystreamer->dio, data, len);
+ break;
+ }
+#endif
if (mystreamer->fd == -1)
break;
@@ -281,6 +408,13 @@ astreamer_extractor_content(astreamer *streamer, astreamer_member *member,
break;
case ASTREAMER_MEMBER_TRAILER:
+#ifdef USE_LIBURING
+ if (mystreamer->dio.fd != -1)
+ {
+ dio_writer_finish(&mystreamer->dio);
+ break;
+ }
+#endif
if (mystreamer->fd == -1)
break;
if (close(mystreamer->fd) != 0)
@@ -463,6 +597,9 @@ astreamer_extractor_finalize(astreamer *streamer)
= (astreamer_extractor *) streamer;
Assert(mystreamer->fd == -1);
+#ifdef USE_LIBURING
+ Assert(mystreamer->dio.fd == -1);
+#endif
}
/*
@@ -474,5 +611,307 @@ astreamer_extractor_free(astreamer *streamer)
astreamer_extractor *mystreamer = (astreamer_extractor *) streamer;
pfree(mystreamer->basepath);
+#ifdef USE_LIBURING
+ dio_writer_destroy(&mystreamer->dio);
+#endif
pfree(mystreamer);
}
+
+#ifdef USE_LIBURING
+/*
+ * IO_uring/liburing uses concept of two rings (submissions and completion).
+ * See io_uring_queue_init(3) and io_uring(7) for more information and
+ * especially https://github.com/axboe/liburing/blob/master/examples/io_uring-cp.c
+ * for nice example on how to use it.
+ */
+
+/* Take (consume) one completion event from the ring */
+static void
+dio_wait_one(dio_writer *dw)
+{
+ int idx;
+ int ret;
+ struct io_uring_cqe *cqe;
+
+ ret = io_uring_wait_cqe(&dw->ring, &cqe);
+ if (ret < 0)
+ {
+ errno = -ret;
+ pg_fatal("could not wait for io_uring_wait_cqe(): %m");
+ }
+
+ idx = (int) io_uring_cqe_get_data64(cqe);
+ if (cqe->res < 0)
+ {
+ errno = -cqe->res;
+ pg_fatal("could not write to file \"%s\": %m", dw->filename);
+ }
+
+ if ((size_t) cqe->res != dw->buflen[idx])
+ {
+ /* short write? */
+ errno = ENOSPC;
+ pg_fatal("could not write to file \"%s\": %m", dw->filename);
+ }
+
+ io_uring_cqe_seen(&dw->ring, cqe);
+ dw->busy[idx] = false;
+ dw->inflight--;
+}
+
+/*
+ * Common routine for flushing (submitting) earlier prepared, but not yet
+ * submitted SQEs.
+ */
+static void
+dio_flush_sq(dio_writer *dw)
+{
+ int ret;
+
+ if (dw->unsubmitted == 0)
+ return;
+
+ ret = io_uring_submit(&dw->ring);
+ if (ret < 0)
+ {
+ errno = -ret;
+ pg_fatal("could not submit io_uring (io_uring_enter(2) failure?): %m");
+ }
+
+ /* Everything was submitted */
+ dw->unsubmitted = 0;
+}
+
+/* Get free buffer index. If none are available, wait for some to finish */
+static int
+dio_get_free_buf(dio_writer *dw)
+{
+ for (;;)
+ {
+ for (int i = 0; i < DIO_NBUF; i++)
+ {
+ if (dw->busy[i] == false)
+ {
+ /*
+ * Mark it busy till we finish the write (technically we need CQE for
+ * this buffer to arrive, and then can mark it as non-busy again).
+ * */
+ dw->busy[i] = true;
+ return i;
+ }
+ }
+ /* All buffers are busy, so we wait for writes to finish */
+ dio_flush_sq(dw);
+ dio_wait_one(dw);
+ }
+}
+
+/*
+ * Prepare the SQE from the buffer with full O_DIRECT write. We really
+ * submit the SQEs to the kernel only (flush them) only once every
+ * couple of times to avoid constant syscall tax (AKA io_uring batching).
+ *
+ * buffer's idx needs to have it's lenth aligned to DIO_ALIGN (O_DIRECT
+ * requirement).
+ */
+static void
+dio_submit(dio_writer *dw, int idx, size_t len)
+{
+ struct io_uring_sqe *sqe;
+
+ sqe = io_uring_get_sqe(&dw->ring);
+ /* this should never happen? liburing/examples uses abort/asserts for this */
+ if (sqe == NULL)
+ pg_fatal("io_uring submission queue full: %m");
+
+ io_uring_prep_write(sqe, dw->fd,
+ dw->pool + (size_t) idx * DIO_BUFSZ,
+ len, dw->offset);
+ io_uring_sqe_set_data64(sqe, idx);
+
+ dw->buflen[idx] = len;
+ dw->offset += len;
+ dw->inflight++;
+ dw->unsubmitted++;
+
+ if (dw->unsubmitted >= DIO_SUBMIT_BATCH)
+ dio_flush_sq(dw);
+}
+
+/*
+ * Start writing to a file through the DIO+AIO path. Returns true if that is
+ * supported (dw->fd is set).
+ *
+ * If prealloc_size is known it can be used to preallocate file which helps
+ * greatly to avoid slow writes with O_DIRECT on some filesystems.
+ */
+static bool
+dio_writer_start(dio_writer *dw, const char *filename, pgoff_t prealloc_size, bool verbose)
+{
+ int fd;
+ int ret;
+ struct stat st;
+
+ fd = open(filename, O_WRONLY | O_CREAT | O_TRUNC | O_DIRECT | PG_BINARY,
+ pg_file_create_mode);
+ if (fd < 0)
+ {
+ if (verbose && !dw->notified)
+ {
+ pg_log_info("could not open \"%s\" with O_DIRECT, using buffered I/O instead: %m",
+ filename);
+ dw->notified = true;
+ }
+ return false;
+ }
+
+ /*
+ * Only regular files can be written with O_DIRECT. However, the /dev/null
+ * device (used with -t client-blackhole mode) and other special files can
+ * be still opened with O_DIRECT, but must go through the buffered path(?).
+ * Ensure we opened regular file before setting up the io_uring, so we
+ * don't allocate the buffer pool just to fall back.
+ */
+ if (fstat(fd, &st) != 0 || !S_ISREG(st.st_mode))
+ {
+ if (verbose && !dw->notified)
+ {
+ pg_log_info("\"%s\" is not a regular file, using buffered I/O instead",
+ filename);
+ dw->notified = true;
+ }
+ close(fd);
+ return false;
+ }
+
+ /* Lazy setup of the io_uring */
+ if (!dw->ring_ready)
+ {
+ ret = io_uring_queue_init(DIO_NBUF, &dw->ring, 0);
+ if (ret != 0)
+ {
+ errno = -ret;
+ /* XXX: shouldn't we warn about it non-verbose mode? */
+ if (verbose && !dw->notified)
+ {
+ pg_log_info("could not set up io_uring, using buffered I/O instead: %m");
+ dw->notified = true;
+ }
+ close(fd);
+ return false;
+ }
+
+ /* O_DIRECT writes require memory aligment */
+ if (posix_memalign((void **) &dw->pool, DIO_ALIGN,
+ (size_t) DIO_NBUF * DIO_BUFSZ) != 0)
+ pg_fatal("unable to allocate aligned memory for direct I/O writes");
+
+ dw->ring_ready = true;
+ if (verbose)
+ pg_log_info("using O_DIRECT with io_uring for large file writes");
+ }
+
+ /* If size is known (in plain mode), preallocate the space */
+#ifdef HAVE_POSIX_FALLOCATE
+ if (prealloc_size > 0)
+ {
+ int rc = posix_fallocate(fd, 0, prealloc_size);
+
+ if (rc == ENOSPC)
+ {
+ errno = rc;
+ pg_fatal("could not preallocate file \"%s\": %m", filename);
+ }
+ }
+#endif
+
+ dw->fd = fd;
+ dw->filename = filename;
+ dw->offset = 0;
+ dw->written = 0;
+ dw->curidx = -1;
+ dw->curlen = 0;
+ return true;
+}
+
+/*
+ * Handle buffering on DIO side (in dio->pool buffers) before submiting
+ * full (big) writes as required by O_DIRECT.
+ */
+static void
+dio_writer_write(dio_writer *dw, const char *data, int len)
+{
+ dw->written += len;
+
+ while (len > 0)
+ {
+ char *buf;
+ int space;
+ int n;
+
+ if (dw->curidx < 0)
+ {
+ dw->curidx = dio_get_free_buf(dw);
+ dw->curlen = 0;
+ }
+
+ buf = dw->pool + (size_t) dw->curidx * DIO_BUFSZ;
+ space = DIO_BUFSZ - dw->curlen;
+ n = Min(space, len);
+ /* append to the pool buffer */
+ memcpy(buf + dw->curlen, data, n);
+ dw->curlen += n;
+ data += n;
+ len -= n;
+
+ if (dw->curlen == DIO_BUFSZ)
+ {
+ dio_submit(dw, dw->curidx, DIO_BUFSZ);
+ dw->curidx = -1;
+ }
+ }
+}
+
+/* Send/wait for remaining in-flight writes, truncate and close the DIO */
+static void
+dio_writer_finish(dio_writer *dw)
+{
+ /* Flush partial buffer */
+ if (dw->curidx >= 0 && dw->curlen > 0)
+ {
+ char *buf = dw->pool + (size_t) dw->curidx * DIO_BUFSZ;
+ /* We need to pad stuff */
+ size_t padded = TYPEALIGN(DIO_ALIGN, dw->curlen);
+
+ memset(buf + dw->curlen, 0, padded - dw->curlen);
+ dio_submit(dw, dw->curidx, padded);
+ dw->curidx = -1;
+ }
+
+ /* Submit any batched writes and wait for them all to finish */
+ dio_flush_sq(dw);
+ while (dw->inflight > 0)
+ dio_wait_one(dw);
+
+ /* Truncate the file to the right size (we could pre-allocate too much) */
+ if (ftruncate(dw->fd, dw->written) != 0)
+ pg_fatal("could not truncate file \"%s\": %m", dw->filename);
+
+ if (close(dw->fd) != 0)
+ pg_fatal("could not close file \"%s\": %m", dw->filename);
+ dw->fd = -1;
+}
+
+/* Free the buffers and the io_uring */
+static void
+dio_writer_destroy(dio_writer *dw)
+{
+ if (dw->ring_ready)
+ {
+ io_uring_queue_exit(&dw->ring);
+ free(dw->pool);
+ dw->ring_ready = false;
+ dw->fd = -1;
+ }
+}
+#endif /* USE_LIBURING */
diff --git a/src/fe_utils/meson.build b/src/fe_utils/meson.build
index 86befca192e..bf508da183c 100644
--- a/src/fe_utils/meson.build
+++ b/src/fe_utils/meson.build
@@ -34,7 +34,7 @@ fe_utils = static_library('libpgfeutils',
c_pch: pch_postgres_fe_h,
include_directories: [postgres_inc, libpq_inc],
c_args: host_system == 'windows' ? ['-DFD_SETSIZE=1024'] : [],
- dependencies: frontend_common_code,
+ dependencies: [frontend_common_code, liburing],
kwargs: default_lib_args + {
'install': install_internal_static_lib,
},
diff --git a/src/include/fe_utils/astreamer.h b/src/include/fe_utils/astreamer.h
index 7206dc0c48b..6eae8ebbb6d 100644
--- a/src/include/fe_utils/astreamer.h
+++ b/src/include/fe_utils/astreamer.h
@@ -210,13 +210,14 @@ astreamer_buffer_until(astreamer *streamer, const char **data, int *len,
* Functions for creating astreamer objects of various types. See the header
* comments for each of these functions for details.
*/
-extern astreamer *astreamer_plain_writer_new(char *pathname, FILE *file);
+extern astreamer *astreamer_plain_writer_new(char *pathname, FILE *file,
+ bool verbose);
extern astreamer *astreamer_gzip_writer_new(char *pathname, FILE *file,
pg_compress_specification *compress);
extern astreamer *astreamer_extractor_new(const char *basepath,
const char *(*link_map) (const char *),
void (*report_output_file) (const char *),
- bool discard_backup);
+ bool discard_backup, bool verbose);
extern astreamer *astreamer_gzip_decompressor_new(astreamer *next);
extern astreamer *astreamer_lz4_compressor_new(astreamer *next,
--
2.43.0
From fc7dbfc8789b1ee047970f715736828d2df70188 Mon Sep 17 00:00:00 2001
From: Debian <admin@ip-10-249-249-30>
Date: Wed, 5 Aug 2026 10:12:22 +0000
Subject: [PATCH v06102026 10/10] pg_basebackup: preallocate DIO writes also in
case of writing tar file format
The DIO from previous commit made tar-format backups slower than the buffered
path it replaced (~410MB/s versus 982MB/s earlier) - the gains were only
visible in the plain format. The problem is related that DIO writes should
avoid extending files, but in case of tar format we do not know the target
(local) tar file that we are producing (unlike like in plain format, where we
know this), so we couldn't preallocate the file properly which tanked write
performance (O_DIRECT writes extending file end up taking exclusive kernel
inode mutex at least on ext4 fs, so apparently even with with plenty of
async DIO requests in flight they all ended up being serialized with depth
queue of 1).
Add dio_fruncate() which - in the tar format case - performs allocation from
time to time, to be ahead of the writes itself. When final output file size
is known (plain format) we just preallocate after open() once.
In case of O_DIRECT without posix_fallocate() working, dio_writer_start()
will now fallback to buffered I/O too as this is going to be faster.
Overall this brings -Ft back to expected 2.3-2.6GB/s.
Author: Jakub Wartak <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/CAKZiRmwwW-hDc3B6ERJB%2BpaX7RNSBcQLheq1KdsTf42cGuRvuA%40mail.gmail.com
---
src/fe_utils/astreamer_file.c | 97 ++++++++++++++++++++++++++++++-----
1 file changed, 83 insertions(+), 14 deletions(-)
diff --git a/src/fe_utils/astreamer_file.c b/src/fe_utils/astreamer_file.c
index d4d34e5d412..efbd9e43a97 100644
--- a/src/fe_utils/astreamer_file.c
+++ b/src/fe_utils/astreamer_file.c
@@ -38,6 +38,7 @@
#define DIO_NBUF 32 /* queue depth, XXX:expose it via getopt? */
#define DIO_MIN_SIZE DIO_BUFSZ /* fsize threshold for activating O_DIRECT writes */
#define DIO_SUBMIT_BATCH 8 /* how many SQEs to batch */
+#define DIO_PREALLOC_CHUNK (128 * 1024 * 1024) /* grow step, unknown size */
/*
* State for the dio/io_uring writer. Used by plain(file) and tar extractors.
@@ -55,10 +56,12 @@ typedef struct dio_writer
const char *filename; /* for error handling */
pgoff_t offset; /* offset of the next write */
pgoff_t written; /* bytes written */
+ pgoff_t allocated; /* file size space preallocated so far */
int curidx; /* buffer being filled, or -1 */
int curlen; /* bytes filled in current buffer */
int inflight; /* prepared writes not yet reaped */
int unsubmitted; /* SQEs prepared but not yet submitted */
+ bool nofalloc; /* fallocate unsupported */
bool notified; /* already logged a fall-back-to-buffered? */
} dio_writer;
#endif
@@ -706,6 +709,59 @@ dio_get_free_buf(dio_writer *dw)
}
}
+/*
+ * Ensure the file has some space ahead allocated to avoid perofrmance issues
+ * with O_DIRECT writes. Returns false if space cannot be allocated and in such
+ * scenario DIO writes should not be used as the are *slower* than buffered
+ * writes (outcome of many performance runs). The reason is that at least on
+ * Linux, async O_DIRECT writes that extend current file size and may end up
+ * allocating space, are queued and the depth drops to 1 due to file extension
+ * /space allocation happening for 1 file.
+ *
+ * This is used from dio_writer_start() and from regular dio_submit().
+ *
+ * End-file size is known in case of plain files (tar format sends
+ * member->size), however in -Ft (tar) writing mode, we do knot know the
+ * final target tar file size, so we grow it by DIO_PREALLOC_CHUNK from time
+ * to time. In case file is overextended, we truncate it back to proper size
+ * in dio_writer_finish().
+ */
+static bool
+dio_fallocate(dio_writer *dw, pgoff_t end)
+{
+#ifdef HAVE_POSIX_FALLOCATE
+ pgoff_t want;
+ int rc;
+
+ if (end <= dw->allocated)
+ return true;
+ if (dw->nofalloc)
+ return false;
+
+ /* Round up to a whole chunk to reduce number of fallocate calls */
+ want = Max(end, dw->allocated + DIO_PREALLOC_CHUNK);
+
+ rc = posix_fallocate(dw->fd, dw->allocated, want - dw->allocated);
+ if (rc != 0)
+ {
+ /* posix_fallocate() does not set errno */
+ errno = rc;
+
+ if (rc == ENOSPC)
+ pg_fatal("could not preallocate file \"%s\": %m", dw->filename);
+
+ dw->nofalloc = true;
+ return false;
+ }
+
+ dw->allocated = want;
+ return true;
+#else
+ dw->nofalloc = true;
+ return false;
+#endif
+}
+
/*
* Prepare the SQE from the buffer with full O_DIRECT write. We really
* submit the SQEs to the kernel only (flush them) only once every
@@ -719,6 +775,12 @@ dio_submit(dio_writer *dw, int idx, size_t len)
{
struct io_uring_sqe *sqe;
+ /*
+ * Keep the allocation ahead of the write cursor. This is no-op when the
+ * file was already preallocated to its final size at open() time (plain format).
+ */
+ dio_fallocate(dw, dw->offset + len);
+
sqe = io_uring_get_sqe(&dw->ring);
/* this should never happen? liburing/examples uses abort/asserts for this */
if (sqe == NULL)
@@ -751,6 +813,9 @@ dio_writer_start(dio_writer *dw, const char *filename, pgoff_t prealloc_size, bo
int fd;
int ret;
struct stat st;
+ /* preallocation in case of DIO also have to be aligned to DIO size */
+ pgoff_t prealloc_size_aligned = TYPEALIGN64(DIO_ALIGN, prealloc_size);
+ pgoff_t prealloc_len = prealloc_size > 0 ? prealloc_size_aligned : DIO_PREALLOC_CHUNK;
fd = open(filename, O_WRONLY | O_CREAT | O_TRUNC | O_DIRECT | PG_BINARY,
pg_file_create_mode);
@@ -811,26 +876,30 @@ dio_writer_start(dio_writer *dw, const char *filename, pgoff_t prealloc_size, bo
pg_log_info("using O_DIRECT with io_uring for large file writes");
}
- /* If size is known (in plain mode), preallocate the space */
-#ifdef HAVE_POSIX_FALLOCATE
- if (prealloc_size > 0)
- {
- int rc = posix_fallocate(fd, 0, prealloc_size);
-
- if (rc == ENOSPC)
- {
- errno = rc;
- pg_fatal("could not preallocate file \"%s\": %m", filename);
- }
- }
-#endif
-
dw->fd = fd;
dw->filename = filename;
dw->offset = 0;
dw->written = 0;
+ dw->allocated = 0;
dw->curidx = -1;
dw->curlen = 0;
+
+ /* Preallocate file to avoid O_DIRECT performance woes */
+ if (!dio_fallocate(dw, prealloc_len))
+ {
+ if (verbose && !dw->notified)
+ {
+ pg_log_info("could not preallocate \"%s\", using buffered I/O instead: %m",
+ filename);
+ dw->notified = true;
+ }
+ dw->fd = -1;
+ close(fd);
+ return false;
+ }
+
+ Assert(dw->inflight == 0);
+ Assert(dw->unsubmitted == 0);
return true;
}
--
2.43.0
From 430b782b2387c7dcdb080dbdff52551609d3a4f3 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 31 Jul 2026 13:51:18 +0200
Subject: [PATCH v06102026 06/10] pg_basebackup: eliminate usage of libc to
coalesce writes() syscalls
glibc is not that efficient when writing lots of data using fwrite(3), which
happens because data received by libpq (recv()) most of time time causes
to generate 2 write calls (4kB one/default BUFSIZE = stat.stblksize due to fs
and the remainder of it using internal glibc's bypass).
We get archive contents already in large chunks, so there is no reason to split
it into multiple write() syscalls. Using setvbuf() is not helpful as it often
causes extra memory copy to be created which makes things even more slow.
astreamer_extractor now tracks an int fd instead of a FILE.
TODO: re-test how much it gains alone? maybe it's all just noise
Author: Jakub Wartak <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/CAKZiRmwwW-hDc3B6ERJB%2BpaX7RNSBcQLheq1KdsTf42cGuRvuA%40mail.gmail.com
---
src/fe_utils/astreamer_file.c | 96 +++++++++++++++++++++--------------
1 file changed, 58 insertions(+), 38 deletions(-)
diff --git a/src/fe_utils/astreamer_file.c b/src/fe_utils/astreamer_file.c
index e4fa2596b08..b867e9489cd 100644
--- a/src/fe_utils/astreamer_file.c
+++ b/src/fe_utils/astreamer_file.c
@@ -15,6 +15,7 @@
#include "postgres_fe.h"
+#include <fcntl.h>
#include <unistd.h>
#include "common/file_perm.h"
@@ -36,7 +37,7 @@ typedef struct astreamer_extractor
const char *(*link_map) (const char *);
void (*report_output_file) (const char *);
char filename[MAXPGPATH];
- FILE *file;
+ int fd;
bool discard_backup;
} astreamer_extractor;
@@ -61,8 +62,10 @@ static void astreamer_extractor_finalize(astreamer *streamer);
static void astreamer_extractor_free(astreamer *streamer);
static void extract_directory(const char *filename, mode_t mode);
static void extract_link(const char *filename, const char *linktarget);
-static FILE *create_file_for_extract(const char *filename, mode_t mode,
- bool discard_backup);
+static int create_file_for_extract(const char *filename, mode_t mode,
+ bool discard_backup);
+static void write_file_range(int fd, const char *filename,
+ const char *data, int len);
static const astreamer_ops astreamer_extractor_ops = {
.content = astreamer_extractor_content,
@@ -117,15 +120,7 @@ astreamer_plain_writer_content(astreamer *streamer,
if (len == 0)
return;
- errno = 0;
- if (fwrite(data, len, 1, mystreamer->file) != 1)
- {
- /* if write didn't set errno, assume problem is no disk space */
- if (errno == 0)
- errno = ENOSPC;
- pg_fatal("could not write to file \"%s\": %m",
- mystreamer->pathname);
- }
+ write_file_range(fileno(mystreamer->file), mystreamer->pathname, data, len);
}
/*
@@ -202,6 +197,7 @@ astreamer_extractor_new(const char *basepath,
streamer->link_map = link_map;
streamer->report_output_file = report_output_file;
streamer->discard_backup = discard_backup;
+ streamer->fd = -1;
return &streamer->base;
}
@@ -223,7 +219,7 @@ astreamer_extractor_content(astreamer *streamer, astreamer_member *member,
switch (context)
{
case ASTREAMER_MEMBER_HEADER:
- Assert(mystreamer->file == NULL);
+ Assert(mystreamer->fd == -1);
if (!path_is_safe_for_extraction(member->pathname))
pg_fatal("tar member has unsafe path name: \"%s\"",
@@ -242,7 +238,7 @@ astreamer_extractor_content(astreamer *streamer, astreamer_member *member,
* Dispatch based on file type.
*/
if (member->is_regular)
- mystreamer->file =
+ mystreamer->fd =
create_file_for_extract(mystreamer->filename,
member->mode,
mystreamer->discard_backup);
@@ -275,27 +271,21 @@ astreamer_extractor_content(astreamer *streamer, astreamer_member *member,
break;
case ASTREAMER_MEMBER_CONTENTS:
- if (mystreamer->file == NULL)
+ if (mystreamer->fd == -1)
break;
- errno = 0;
- if (len > 0 && fwrite(data, len, 1, mystreamer->file) != 1)
- {
- /* if write didn't set errno, assume problem is no disk space */
- if (errno == 0)
- errno = ENOSPC;
- pg_fatal("could not write to file \"%s\": %m",
- mystreamer->filename);
- }
+ if (len > 0)
+ write_file_range(mystreamer->fd, mystreamer->filename,
+ data, len);
break;
case ASTREAMER_MEMBER_TRAILER:
- if (mystreamer->file == NULL)
+ if (mystreamer->fd == -1)
break;
- if (fclose(mystreamer->file) != 0)
+ if (close(mystreamer->fd) != 0)
pg_fatal("could not close file \"%s\": %m",
mystreamer->filename);
- mystreamer->file = NULL;
+ mystreamer->fd = -1;
break;
case ASTREAMER_ARCHIVE_TRAILER:
@@ -380,23 +370,28 @@ extract_link(const char *filename, const char *linktarget)
/*
* Create a regular file.
*
- * Return the resulting handle so we can write the content to the file.
+ * Return an open file descriptor so we can write the content to the file.
+ *
+ * We intentionally use a raw file descriptor to get unbuffered write()
+ * and avoid potentiall libc interference.
*/
-static FILE *
-create_file_for_extract(const char *filename, mode_t mode, bool discard_backup)
+static int
+create_file_for_extract(const char *filename, mode_t mode,
+ bool discard_backup)
{
- FILE *file;
+ int fd;
if (discard_backup)
{
- file = fopen(DEVNULL, "wb");
- if (file == NULL)
+ fd = open(DEVNULL, O_WRONLY | PG_BINARY, 0);
+ if (fd < 0)
pg_fatal("could not open file \"%s\": %m", DEVNULL);
- return file;
+ return fd;
}
- file = fopen(filename, "wb");
- if (file == NULL)
+ fd = open(filename, O_WRONLY | O_CREAT | O_TRUNC | PG_BINARY,
+ pg_file_create_mode);
+ if (fd < 0)
pg_fatal("could not create file \"%s\": %m", filename);
#ifndef WIN32
@@ -405,7 +400,32 @@ create_file_for_extract(const char *filename, mode_t mode, bool discard_backup)
filename);
#endif
- return file;
+ return fd;
+}
+
+/*
+ * Wrapper for safely writing chunk of archive. Single write() is not
+ * guaranteed to consume the whole, so we loop until all is on the disk.
+ */
+static void
+write_file_range(int fd, const char *filename, const char *data, int len)
+{
+ while (len > 0)
+ {
+ ssize_t written;
+
+ errno = 0;
+ written = write(fd, data, len);
+ if (written <= 0)
+ {
+ if (errno == 0)
+ errno = ENOSPC;
+ pg_fatal("could not write to file \"%s\": %m", filename);
+ }
+
+ data += written;
+ len -= written;
+ }
}
/*
@@ -419,7 +439,7 @@ astreamer_extractor_finalize(astreamer *streamer)
astreamer_extractor *mystreamer PG_USED_FOR_ASSERTS_ONLY
= (astreamer_extractor *) streamer;
- Assert(mystreamer->file == NULL);
+ Assert(mystreamer->fd == -1);
}
/*
--
2.43.0
