Hi, COPY TO from a table deforms every attribute of each row, even when its column list ends well before the last column. The attached patch deforms only up to the highest attribute that is sent. Text, CSV and binary output read only the listed attributes, so their output does not change. JSON output still deforms the whole row, and so does the partition path that converts tuples to the root table's row type.
slot_getallattrs() walks each attribute to find its offset and stores a Datum and a null flag for it. With the patch, the walk stops at the highest exported attribute. This helps exports of leading columns from wide tables. One in-core caller is the initial table synchronization of a publication with a column list: the publisher runs COPY t (published columns) TO STDOUT. pg_dump lists every column, so it gains only when a table's last columns are generated or dropped. Measurements on a 300,000-row table with 60 columns (234 MB) of int, bigint, numeric, text, varchar, timestamptz and bool, some half NULL. COPY ... TO '/dev/null', warm shared_buffers, -O2 build without assertions, x86-64 VM with 8 vCPUs. Median ms per COPY over alternating unpatched/patched pairs (3 pairs for narrow exports, 11 for the rest): columns exported unpatched patched first 5 of 60, CSV 290 191 -34% first 5 of 60, text 252 174 -31% column 30 of 60, CSV 162 127 -22% id and column 60, CSV 170 167 no change all 60, CSV 1257 1266 no change pgbench_accounts, all 287 286 no change Run-to-run spread was about 10%, so differences under 5% are noise. For a full row the patched code does the same deforming as before. With the patch, COPY t (first 5 columns) costs about the same as COPY (SELECT first 5 columns FROM t), which took 189 and 195 ms. There is no new regression test, because the output does not change. With assertions enabled, the regress suite and the subscription TAP tests (including 031_column_list) pass, and pgindent is clean. The output of 15 COPY TO commands matched an unpatched build byte for byte: column lists in and out of order, a dropped column, columns added with a default, a zero-column table, a partitioned table with a reordered partition, and the text, CSV, binary and JSON formats. The patch is against master. It is a performance change, so it is not meant for back-patching. Tool-use disclosure: Perfloop found the issue and made the patch. Human owner: me, Tomás Senart.
From 875237b7753ae83b1f3ad611da67408a92f2afd2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tom=C3=A1s=20Senart?= <[email protected]> Date: Sun, 15 Nov 2026 20:00:00 +0000 Subject: [PATCH v1] Avoid deforming unused trailing columns in COPY TO COPY TO from a table deformed every attribute of each row, even when its column list ends well before the last column. Deform only up to the highest attribute that is sent. Text, CSV and binary output read only the listed attributes. JSON output still deforms the whole row, and so does the partition path that converts tuples to the root table's row type. This helps exports of leading columns from wide tables, including the COPY that the publisher runs for the initial table synchronization of a publication with a column list. --- src/backend/commands/copyto.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c index 9ca06ed..6dfa532 100644 --- a/src/backend/commands/copyto.c +++ b/src/backend/commands/copyto.c @@ -87,6 +87,7 @@ typedef struct CopyToStateData Relation rel; /* relation to copy to */ QueryDesc *queryDesc; /* executable query to copy from */ List *attnumlist; /* integer list of attnums to copy */ + int max_attnum; /* highest attribute to fetch from the slot */ char *filename; /* filename, or NULL for STDOUT */ bool is_program; /* is 'filename' a program to popen? */ bool json_row_delim_needed; /* need delimiter before next row */ @@ -1326,8 +1327,13 @@ DoCopyTo(CopyToState cstate) cstate->routine->CopyToOutFunc(cstate, attr->atttypid, &cstate->out_functions[attnum - 1]); + cstate->max_attnum = Max(cstate->max_attnum, attnum); } + /* JSON output may need the whole row to construct a composite datum. */ + if (cstate->opts.format == COPY_FORMAT_JSON) + cstate->max_attnum = num_phys_attrs; + /* * Create a temporary memory context that we can reset once per row to * recover palloc'd memory. This avoids any problems with leaks inside @@ -1420,8 +1426,8 @@ CopyRelationTo(CopyToState cstate, Relation rel, Relation root_rel, uint64 *proc copyslot = execute_attr_map_slot(map, slot, root_slot); else { - /* Deconstruct the tuple */ - slot_getallattrs(slot); + /* Deconstruct the prefix needed for output. */ + slot_getsomeattrs(slot, cstate->max_attnum); copyslot = slot; } @@ -1457,8 +1463,8 @@ CopyOneRowTo(CopyToState cstate, TupleTableSlot *slot) MemoryContextReset(cstate->rowcontext); oldcontext = MemoryContextSwitchTo(cstate->rowcontext); - /* Make sure the tuple is fully deconstructed */ - slot_getallattrs(slot); + /* Fetch only the prefix containing the attributes needed for output. */ + slot_getsomeattrs(slot, cstate->max_attnum); cstate->routine->CopyToOneRow(cstate, slot); -- 2.55.0
