Hello Masahiko, Sutou-san, everyone,
You closed your last message with:
> I'll verify that the new API works well with an experimental custom
> copy format extension.
I went ahead and did that, since the entry has been sitting since June.
I wrote a real custom format against v3 - newline-delimited JSON, both
directions, attached - and the short answer is that the API shape works,
but an extension cannot currently implement a format with it.
Everything below is on master 09a579abaca, v3 applied (0004 needs a
one-line rebase: typedefs.list now has TestCustomScanState where the
hunk expects to insert).
1. An extension cannot move any bytes
--------------------------------------
The callbacks are fine, but the functions that talk to the COPY source
and destination are all static:
copyto.c: CopySendData, CopySendEndOfRow,
CopySendTextLikeEndOfRow
copyfromparse.c: CopyGetData, CopyReadLine
copy_state.h exports no functions at all. So a format can be registered
and its callbacks called, but CopyToOneRow has no way to hand its bytes
to the destination, and CopyFromOneRow has no way to pull bytes from
the source. The destination can be a file, a PROGRAM, the frontend or a
callback, and only CopySendEndOfRow/CopyGetData know the difference;
reimplementing that in every extension is not an option.
I think this is why test_copy_custom_format only emits NOTICEs: it is
the most an extension can do today, and that is also why the test
module cannot catch this.
Attached nocfbot-export-copy-io.diff.txt exports the three that a format
actually needs (CopySendData, CopySendEndOfRow, CopyGetData) and
declares them in copyapi.h. With that diff on top of v3, the ndjson
extension compiles and works:
COPY ndj TO STDOUT WITH (FORMAT ndjson)
{"id":"1","name":"hello","amount":"1.25","ok":"t","ts":"2026-01-31"}
{"id":"2","name":"with \"quotes\" and \\ backslash",...}
{"id":"3","name":null,"amount":null,"ok":null,"ts":null}
and a full round trip through a file returns the data unchanged:
quotes, backslashes, embedded tabs and newlines, NULLs and a leap-year
date all survive, and both EXCEPT ALL checks come back empty. A column
list is honoured on both sides, and keys missing from a line come back
as NULL.
Two smaller things I hit while writing it, both worth a line of
documentation in copyapi.h rather than a code change:
* CopySendEndOfRow does not terminate the row. The built-in text/CSV
formats go through CopySendTextLikeEndOfRow for that, which stays
static, so a line-oriented custom format has to append its own "\n".
Nothing says so, and there is no example to copy from.
* The format gets its state through cstate->format_private, which is
never mentioned in the API comments.
2. The performance question
----------------------------
This thread has been circling performance since 2023, and in December
Sutou-san's six runs showed no reproducible trend while you noted that
0001-from-binary and 0006-to-binary looked slower in all six.
I think the reason those results never settle is that the effect being
chased is smaller than the measurement noise. What v3 changes on the
built-in path is in ProcessCopyOptions(), which runs once per COPY
command, not per row - so a large COPY cannot see it by construction.
I measured anyway, 1M rows, {1,10,100} int columns, text/csv/binary,
COPY TO and FROM, 7 repetitions, pinned to one core (scripts attached).
The first pass suggested text COPY FROM with 100 columns was 11% slower
under v3. That turned out to be an artifact of my own harness: master
always ran first within a repetition. Re-running the suspicious cases
in both orders, 10 repetitions each:
text FROM 100 cols, master first: +4.9%
text FROM 100 cols, v3 first: +2.7%
binary FROM 100 cols, master first: -2.2%
binary FROM 100 cols, v3 first: +1.3%
binary changes sign with the order, so that one was the harness. text
keeps its sign, so something is there. I then built a tree with only
0001 applied - the patch that just moves the structs into copy_state.h
and renames the COPY_FILE/COPY_FRONTEND enums, with no execution-path
change at all - and measured it the same way:
text FROM 100 cols, master first: +2.9% v-first: +0.5%
binary FROM 100 cols, master first: +4.3% v-first: +2.6%
A patch that only moves declarations between headers cannot cost 3-4%
of a COPY. So what these numbers are showing is code layout and
measurement noise, not the cost of the API - which also explains the
contradictory heatmaps from earlier in the thread. I would suggest not
letting this block the design any further; if someone wants a real
answer it needs a harness built for it, not bigger COPYs.
I did not benchmark a custom format against a built-in one, since that
compares two different amounts of work.
3. v3 does not pass its own test
---------------------------------
make -C src/test/modules/test_copy_custom_format check -> FAIL
and it fails on a plain v3 tree, with none of my changes applied, so
this is not something I introduced. The diff is this part of the
expected output never happening:
COPY copy_data FROM stdin WITH (format 'test_format',
disallow_freeze true); -- OK
NOTICE: CopyFromInFunc: attribute: smallint
...
The cause is the line just before it in the test script:
COPY copy_data FROM stdin WITH (format 'test_format', freeze true,
disallow_freeze true); -- ERROR
Reproduced by hand: the server has already switched the connection to
copy-in mode by the time the validation callback raises, so psql
swallows the *next* line - the second COPY - as data for the failed
one, together with the terminating \. The second statement never runs.
Two statements, both COPY ... FROM stdin, one of which is meant to
fail: the error case needs to go last, or the OK case needs its own
input. Worth deciding whether the validation callback should run
before the connection flips to copy-in, which would also make this
test behave the way it reads.
4. Also
--------
make check, src/test/modules (other than the above) and
src/bin/pg_dump pass with v3.
Sutou-san asked for CopyFormatIsBuiltin() back in June; v3 has
CopyFormatIsBuiltins(), still plural for a macro that returns a
boolean.
Happy to turn the ndjson module into a proper test module for the
series if you think it is worth having something in the tree that
actually moves data through the API. It is attached as plain .txt for
now.
Regards,
Manu
/*--------------------------------------------------------------------------
*
* test_copy_ndjson.c
* A REAL custom COPY format (newline-delimited JSON), written
against
* the API proposed in the "Make COPY format extendable" thread,
to see
* whether an extension can actually implement a format with it.
*
* Unlike test_copy_custom_format, this one moves data: COPY TO writes one
* JSON object per row and COPY FROM reads it back. That is what exercises
* the parts of the API an extension really needs.
*
* Portions Copyright (c) 2026, PostgreSQL Global Development Group
*
* -------------------------------------------------------------------------
*/
#include "postgres.h"
#include "commands/copy.h"
#include "commands/copy_state.h"
#include "commands/copyapi.h"
#include "catalog/pg_proc_d.h"
#include "mb/pg_wchar.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/json.h"
#include "utils/jsonfuncs.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
PG_MODULE_MAGIC;
typedef struct NdjsonToState
{
TupleDesc tupDesc;
StringInfoData row;
} NdjsonToState;
typedef struct NdjsonFromState
{
TupleDesc tupDesc;
StringInfoData line;
bool eof;
} NdjsonFromState;
/* lookup info for json_object_field_text(), filled in at CopyFromStart */
static FmgrInfo ndjson_field_finfo;
/* ---------------------------------------------------------------- COPY TO */
static void
NdjsonToOutFunc(CopyToState cstate, Oid atttypid, FmgrInfo *finfo)
{
Oid func_oid;
bool is_varlena;
getTypeOutputInfo(atttypid, &func_oid, &is_varlena);
fmgr_info(func_oid, finfo);
}
static void
NdjsonToStart(CopyToState cstate, TupleDesc tupDesc)
{
NdjsonToState *st = palloc0_object(NdjsonToState);
st->tupDesc = tupDesc;
initStringInfo(&st->row);
cstate->format_private = (void *) st;
}
static void
NdjsonToOneRow(CopyToState cstate, TupleTableSlot *slot)
{
NdjsonToState *st = (NdjsonToState *) cstate->format_private;
ListCell *lc;
bool first = true;
resetStringInfo(&st->row);
appendStringInfoChar(&st->row, '{');
foreach(lc, cstate->attnumlist)
{
int attnum = lfirst_int(lc);
Form_pg_attribute attr = TupleDescAttr(st->tupDesc, attnum - 1);
Datum value = slot->tts_values[attnum - 1];
bool isnull = slot->tts_isnull[attnum - 1];
if (!first)
appendStringInfoChar(&st->row, ',');
first = false;
escape_json(&st->row, NameStr(attr->attname));
appendStringInfoChar(&st->row, ':');
if (isnull)
appendStringInfoString(&st->row, "null");
else
{
char *str =
OutputFunctionCall(&cstate->out_functions[attnum - 1],
value);
escape_json(&st->row, str);
}
}
appendStringInfoChar(&st->row, '}');
/*
* CopySendEndOfRow is the raw primitive and does not terminate the row;
* the built-in text/CSV formats go through CopySendTextLikeEndOfRow for
* that, which is static too. So a line-oriented custom format has to
* emit its own terminator. Worth documenting in copyapi.h: nothing in
* the API says so, and the bundled test module never writes any data,
so
* there is no example to copy from.
*/
appendStringInfoChar(&st->row, '\n');
/*
* Here is the whole point of this module: an extension has to be able
to
* hand these bytes to whatever destination the COPY is going to (file,
* PROGRAM, frontend, callback). Only CopySendData/CopySendEndOfRow
know
* how to do that, and in the posted patch both are static in copyto.c.
*/
CopySendData(cstate, st->row.data, st->row.len);
CopySendEndOfRow(cstate);
}
static void
NdjsonToEnd(CopyToState cstate)
{
}
static const CopyToRoutine NdjsonToRoutine = {
.CopyToOutFunc = NdjsonToOutFunc,
.CopyToStart = NdjsonToStart,
.CopyToOneRow = NdjsonToOneRow,
.CopyToEnd = NdjsonToEnd,
};
/* -------------------------------------------------------------- COPY FROM */
static void
NdjsonFromInFunc(CopyFromState cstate, Oid atttypid, FmgrInfo *finfo,
Oid *typioparam)
{
Oid func_oid;
getTypeInputInfo(atttypid, &func_oid, typioparam);
fmgr_info(func_oid, finfo);
}
static void
NdjsonFromStart(CopyFromState cstate, TupleDesc tupDesc)
{
NdjsonFromState *st = palloc0_object(NdjsonFromState);
st->tupDesc = tupDesc;
initStringInfo(&st->line);
cstate->format_private = (void *) st;
fmgr_info(F_JSON_OBJECT_FIELD_TEXT, &ndjson_field_finfo);
}
/*
* Read one newline-terminated line from the COPY source.
*
* Same story as on the TO side: the source can be a file, a program or the
* frontend, and CopyGetData is the only thing that knows the difference.
* It is static in copyfromparse.c.
*/
static bool
ndjson_read_line(CopyFromState cstate, NdjsonFromState *st)
{
char c;
resetStringInfo(&st->line);
for (;;)
{
if (CopyGetData(cstate, &c, 1, 1) != 1)
return st->line.len > 0; /* last line without
newline */
if (c == '\n')
return true;
if (c == '\r')
continue;
appendStringInfoChar(&st->line, c);
}
}
static bool
NdjsonFromOneRow(CopyFromState cstate, ExprContext *econtext,
Datum *values, bool *nulls)
{
NdjsonFromState *st = (NdjsonFromState *) cstate->format_private;
Datum json;
ListCell *lc;
if (!ndjson_read_line(cstate, st))
return false;
/* an extension is free to use whatever parser it likes; json_in is
handy */
json = DirectFunctionCall1(json_in, CStringGetDatum(st->line.data));
memset(nulls, true, st->tupDesc->natts * sizeof(bool));
foreach(lc, cstate->attnumlist)
{
int attnum = lfirst_int(lc);
Form_pg_attribute attr = TupleDescAttr(st->tupDesc, attnum - 1);
Datum field;
bool isnull;
LOCAL_FCINFO(fcinfo, 2);
/*
* json_object_field_text returns NULL for a missing key or a
JSON
* null, so it cannot be called through DirectFunctionCall.
*/
InitFunctionCallInfoData(*fcinfo, &ndjson_field_finfo, 2,
InvalidOid,
NULL, NULL);
fcinfo->args[0].value = json;
fcinfo->args[0].isnull = false;
fcinfo->args[1].value =
CStringGetTextDatum(NameStr(attr->attname));
fcinfo->args[1].isnull = false;
field = FunctionCallInvoke(fcinfo);
isnull = fcinfo->isnull;
if (isnull)
{
nulls[attnum - 1] = true;
values[attnum - 1] = (Datum) 0;
}
else
{
char *str = TextDatumGetCString(field);
values[attnum - 1] =
InputFunctionCall(&cstate->in_functions[attnum - 1],
str,
cstate->typioparams[attnum - 1],
attr->atttypmod);
nulls[attnum - 1] = false;
}
}
return true;
}
static void
NdjsonFromEnd(CopyFromState cstate)
{
}
static const CopyFromRoutine NdjsonFromRoutine = {
.CopyFromInFunc = NdjsonFromInFunc,
.CopyFromStart = NdjsonFromStart,
.CopyFromOneRow = NdjsonFromOneRow,
.CopyFromEnd = NdjsonFromEnd,
};
void
_PG_init(void)
{
RegisterCopyCustomFormat("ndjson", &NdjsonToRoutine, &NdjsonFromRoutine,
NULL, NULL);
}
-- A real custom COPY format: newline-delimited JSON, round trip.
LOAD 'test_copy_ndjson';
CREATE TABLE ndj (id int, name text, amount numeric, ok bool, ts date);
INSERT INTO ndj VALUES
(1, 'hello', 1.25, true, '2026-01-31'),
(2, 'with "quotes" and \ backslash', -0.5, false, '1999-12-31'),
(3, NULL, NULL, NULL, NULL),
(4, E'tab\there and newline\nthere', 1e6, true, '2000-02-29');
-- COPY TO: one JSON object per row
COPY ndj TO STDOUT WITH (FORMAT ndjson);
-- round trip through a file
COPY ndj TO '/tmp/ndj_test.ndjson' WITH (FORMAT ndjson);
CREATE TABLE ndj_back (LIKE ndj);
COPY ndj_back FROM '/tmp/ndj_test.ndjson' WITH (FORMAT ndjson);
-- the data must survive unchanged
SELECT count(*) AS roundtripped FROM ndj_back;
SELECT * FROM ndj EXCEPT ALL SELECT * FROM ndj_back;
SELECT * FROM ndj_back EXCEPT ALL SELECT * FROM ndj;
-- a column list must be honoured on both sides
COPY ndj (id, name) TO STDOUT WITH (FORMAT ndjson);
-- COPY FROM with a column list: missing keys become NULL
TRUNCATE ndj_back;
COPY ndj (id, name) TO '/tmp/ndj_partial.ndjson' WITH (FORMAT ndjson);
COPY ndj_back (id, name) FROM '/tmp/ndj_partial.ndjson' WITH (FORMAT ndjson);
SELECT id, name, amount IS NULL AS amount_null FROM ndj_back ORDER BY id;
DROP TABLE ndj, ndj_back;
diff --git a/src/backend/commands/copyfromparse.c
b/src/backend/commands/copyfromparse.c
index df6ea33ff14..6b40a379ee5 100644
--- a/src/backend/commands/copyfromparse.c
+++ b/src/backend/commands/copyfromparse.c
@@ -163,7 +163,7 @@ static pg_always_inline bool
NextCopyFromRawFieldsInternal(CopyFromState cstate,
/* Low-level communications functions */
-static int CopyGetData(CopyFromState cstate, void *databuf,
+int CopyGetData(CopyFromState cstate, void *databuf,
int minread, int maxread);
static inline bool CopyGetInt32(CopyFromState cstate, int32 *val);
static inline bool CopyGetInt16(CopyFromState cstate, int16 *val);
@@ -245,7 +245,7 @@ ReceiveCopyBinaryHeader(CopyFromState cstate)
*
* NB: no data conversion is applied here.
*/
-static int
+int
CopyGetData(CopyFromState cstate, void *databuf, int minread, int maxread)
{
int bytesread = 0;
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index af2aab18234..fb9013e8622 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -82,10 +82,10 @@ static void CopyToBinaryEnd(CopyToState cstate);
/* Low-level communications functions */
static void SendCopyBegin(CopyToState cstate);
static void SendCopyEnd(CopyToState cstate);
-static void CopySendData(CopyToState cstate, const void *databuf, int
datasize);
+/* exported for custom formats, see copyapi.h */
static void CopySendString(CopyToState cstate, const char *str);
static void CopySendChar(CopyToState cstate, char c);
-static void CopySendEndOfRow(CopyToState cstate);
+
static void CopySendTextLikeEndOfRow(CopyToState cstate);
static void CopySendInt32(CopyToState cstate, int32 val);
static void CopySendInt16(CopyToState cstate, int16 val);
@@ -513,7 +513,7 @@ SendCopyEnd(CopyToState cstate)
* NB: no data conversion is applied by these functions
*----------
*/
-static void
+void
CopySendData(CopyToState cstate, const void *databuf, int datasize)
{
appendBinaryStringInfo(cstate->fe_msgbuf, databuf, datasize);
@@ -531,7 +531,7 @@ CopySendChar(CopyToState cstate, char c)
appendStringInfoCharMacro(cstate->fe_msgbuf, c);
}
-static void
+void
CopySendEndOfRow(CopyToState cstate)
{
StringInfo fe_msgbuf = cstate->fe_msgbuf;
diff --git a/src/include/commands/copyapi.h b/src/include/commands/copyapi.h
index f1924424df8..361a707fdef 100644
--- a/src/include/commands/copyapi.h
+++ b/src/include/commands/copyapi.h
@@ -145,4 +145,17 @@ extern void RegisterCopyCustomFormat(const char *name,
const CopyToRoutine *to,
extern const CopyCustomFormatEntry *GetCopyCustomFormatRoutines(const char
*name);
+/*
+ * Low-level I/O for custom formats.
+ *
+ * A format implementation has to move bytes to and from whatever the COPY is
+ * attached to (a file, a PROGRAM, the frontend, or a callback), and only these
+ * know how to do that. Without them an extension can register a format but
+ * cannot implement one.
+ */
+extern void CopySendData(CopyToState cstate, const void *databuf, int
datasize);
+extern void CopySendEndOfRow(CopyToState cstate);
+extern int CopyGetData(CopyFromState cstate, void *databuf,
+ int minread, int maxread);
+
#endif /* COPYAPI_H */
#!/usr/bin/env bash
# #4681: el rendimiento es LA preocupacion que tiene trabado este hilo.
#
# Lo que cambia v3 en el camino de los formatos built-in esta en
# ProcessCopyOptions(), que corre UNA VEZ POR COMANDO, no por fila. Asi que
# se miden las dos cosas:
#
# A) COPY grande (1M filas): mide el bucle de filas. Si v3 no lo toca,
# aca no deberia verse nada.
# B) muchos COPY chicos (N comandos de 1 fila): mide el costo POR COMANDO,
# que es el unico lugar donde el cambio podria aparecer.
#
# Las corridas se ALTERNAN master/v3 para que la deriva de la maquina no se
# le cargue a uno de los dos, y todo corre pineado al mismo core.
set -u
BASE=$HOME/pg4681
CORE=${CORE:-3}
REPS=${REPS:-7}
FILAS=${FILAS:-1000000}
CHICOS=${CHICOS:-3000}
OUT=$BASE/bench.csv
arranca() { # $1=arbol $2=puerto
local w=$1 p=$2
rm -rf "$BASE/data_$w"
"$BASE/i-$w/bin/initdb" -D "$BASE/data_$w" -U postgres --no-sync -A trust >
/dev/null 2>&1
taskset -c "$CORE" "$BASE/i-$w/bin/pg_ctl" -D "$BASE/data_$w" \
-o "-p $p -c fsync=off -c synchronous_commit=off -c full_page_writes=off -c
max_wal_size=8GB -c shared_buffers=2GB" \
-l "$BASE/log_$w.log" -w start > /dev/null 2>&1
}
para() { "$BASE/i-$1/bin/pg_ctl" -D "$BASE/data_$1" -w stop > /dev/null 2>&1; }
psql_() { local w=$1 p=$2; shift 2; taskset -c "$CORE" "$BASE/i-$w/bin/psql" -p
"$p" -U postgres -qtAX "$@"; }
# $1=arbol $2=puerto $3=ncols $4=formato $5=op -> ms
mide() {
local w=$1 p=$2 n=$3 fmt=$4 op=$5
local f="$BASE/dump_${n}_${fmt}.dat"
if [ "$op" = TO ]; then
psql_ "$w" "$p" -c "\\timing on" -c "COPY t$n TO '$f' WITH (FORMAT $fmt)"
2>/dev/null \
| grep -oP 'Time: \K[0-9.]+' | tail -1
else
psql_ "$w" "$p" -c "TRUNCATE t${n}_in" -c "\\timing on" \
-c "COPY t${n}_in FROM '$f' WITH (FORMAT $fmt)" 2>/dev/null \
| grep -oP 'Time: \K[0-9.]+' | tail -1
fi
}
prepara() { # tablas y datos
local w=$1 p=$2
for n in 1 10 100; do
local cols="" vals=""
for i in $(seq 1 "$n"); do cols="$cols,c$i int";
vals="$vals,(random()*1000000)::int"; done
cols=${cols#,}; vals=${vals#,}
psql_ "$w" "$p" -c "DROP TABLE IF EXISTS t$n, t${n}_in" \
-c "CREATE TABLE t$n ($cols)" -c "CREATE TABLE t${n}_in ($cols)" \
-c "INSERT INTO t$n SELECT $vals FROM generate_series(1,$FILAS)" >
/dev/null
done
psql_ "$w" "$p" -c "DROP TABLE IF EXISTS chico" -c "CREATE TABLE chico (a
int)" > /dev/null
}
echo "arbol,rep,ncols,formato,op,ms" > "$OUT"
arranca master 55501; prepara master 55501
arranca v3 55502; prepara v3 55502
for r in $(seq 1 "$REPS"); do
for w in master v3; do
p=55501; [ "$w" = v3 ] && p=55502
for n in 1 10 100; do
for fmt in text csv binary; do
for op in TO FROM; do
ms=$(mide "$w" "$p" "$n" "$fmt" "$op")
echo "$w,$r,$n,$fmt,$op,${ms:-NA}" >> "$OUT"
done
done
done
# B) costo POR COMANDO: N copys de una fila
ini=$(date +%s.%N)
psql_ "$w" "$p" -c "DO \$\$ BEGIN FOR i IN 1..$CHICOS LOOP
EXECUTE 'COPY chico FROM PROGRAM ''echo 1'' WITH (FORMAT text)'; END
LOOP; END \$\$" > /dev/null
fin=$(date +%s.%N)
echo "$w,$r,0,percmd,FROM,$(echo "($fin-$ini)*1000" | bc)" >> "$OUT"
done
echo " rep $r/$REPS listo"
done
para master; para v3
echo "-> $OUT"
#!/usr/bin/env bash
# Control del sesgo de ORDEN.
#
# En bench.sh, dentro de cada repeticion master corre siempre ANTES que v3.
# Si la maquina se calienta, o el cache de archivos favorece a quien corre
# segundo, ese efecto se le carga siempre al mismo arbol y aparece como si
# fuera del patch.
#
# Aca se corren los casos sospechosos (100 columnas, COPY FROM) en las DOS
# ordenes. Si el signo del delta se mantiene, es del patch; si se da vuelta,
# es del orden.
set -u
BASE=$HOME/pg4681
CORE=${CORE:-3}
REPS=${REPS:-10}
OUT=$BASE/orden.csv
arranca() {
local w=$1 p=$2
rm -rf "$BASE/dord_$w"
"$BASE/i-$w/bin/initdb" -D "$BASE/dord_$w" -U postgres --no-sync -A trust >
/dev/null 2>&1
taskset -c "$CORE" "$BASE/i-$w/bin/pg_ctl" -D "$BASE/dord_$w" \
-o "-p $p -c fsync=off -c synchronous_commit=off -c full_page_writes=off -c
max_wal_size=8GB -c shared_buffers=2GB" \
-l "$BASE/logord_$w.log" -w start > /dev/null 2>&1
}
psql_() { local w=$1 p=$2; shift 2; taskset -c "$CORE" "$BASE/i-$w/bin/psql" -p
"$p" -U postgres -qtAX "$@"; }
prep() {
local w=$1 p=$2 cols="" vals=""
for i in $(seq 1 100); do cols="$cols,c$i int";
vals="$vals,(random()*1000000)::int"; done
psql_ "$w" "$p" -c "CREATE TABLE t100 (${cols#,})" -c "CREATE TABLE t100_in
(${cols#,})" \
-c "INSERT INTO t100 SELECT ${vals#,} FROM generate_series(1,1000000)" >
/dev/null
for fmt in text binary; do
psql_ "$w" "$p" -c "COPY t100 TO '$BASE/ord_$fmt.dat' WITH (FORMAT $fmt)" >
/dev/null
done
}
mide() { # $1=arbol $2=puerto $3=formato -> ms de COPY FROM
psql_ "$1" "$2" -c "TRUNCATE t100_in" -c "\\timing on" \
-c "COPY t100_in FROM '$BASE/ord_$3.dat' WITH (FORMAT $3)" 2>/dev/null \
| grep -oP 'Time: \K[0-9.]+' | tail -1
}
arranca master 55521; prep master 55521
arranca v3 55522; prep v3 55522
echo "orden,arbol,rep,formato,ms" > "$OUT"
for r in $(seq 1 "$REPS"); do
for fmt in text binary; do
# orden A: master primero
a=$(mide master 55521 "$fmt"); echo "A,master,$r,$fmt,$a" >> "$OUT"
b=$(mide v3 55522 "$fmt"); echo "A,v3,$r,$fmt,$b" >> "$OUT"
# orden B: v3 primero
c=$(mide v3 55522 "$fmt"); echo "B,v3,$r,$fmt,$c" >> "$OUT"
d=$(mide master 55521 "$fmt"); echo "B,master,$r,$fmt,$d" >> "$OUT"
done
echo " rep $r/$REPS"
done
"$BASE/i-master/bin/pg_ctl" -D "$BASE/dord_master" -w stop > /dev/null 2>&1
"$BASE/i-v3/bin/pg_ctl" -D "$BASE/dord_v3" -w stop > /dev/null 2>&1
echo "-> $OUT"
