Hi, On Mon, 7 Sept 2026 at 19:29, Nazir Bilal Yavuz <[email protected]> wrote: > > When fewer than one SIMD vector of bytes (16 in this case) remain > buffered, CopyReadLineTextSIMDHelper() refills the input before > falling back to scalar parsing. If those buffered bytes already > contain a complete '\.' (end-of-copy marker) and the source is an open > FIFO with no more data, the refill blocks, so COPY fails to recognize > the marker until more data arrives or the writer closes the FIFO. > Reproducer SQL script is attached, you can see that this causes a > hang. > > One potential fix is checking for an end-of-copy marker when fewer > than one SIMD vector of bytes remains in the buffer and we decide to > load more data into it. I haven't benchmarked this solution yet but it > could potentially cause a slowdown when we load data in smaller > chunks. Otherwise, I don't think this solution will cause a slowdown. > I am planning to work on this tomorrow. > > Any opinions on the bug or the potential solution?
Here is an attempt to solve this problem. I ran Manni's script and saw a 1-2% slowdown on TEXT-wide inputs. It is still faster compared to the version without the SIMD patch; the slowdown is relative to the current master branch (which includes the SIMD patch). -- Regards, Nazir Bilal Yavuz Microsoft
From cef5dc405842841ca4355f9b0607e0919c349841 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Tue, 8 Sep 2026 13:06:19 +0300 Subject: [PATCH v1] Fix COPY SIMD refill blocking on buffered end markers Check short text tails for a backslash before refilling, leaving marker recognition to the scalar parser. This avoids blocking on an open FIFO with a buffered end marker while preserving SIMD refills for long lines. Fixes a regression introduced by e0a3a3fd536. --- src/backend/commands/copyfromparse.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/src/backend/commands/copyfromparse.c b/src/backend/commands/copyfromparse.c index 98bf30ef2e7..2001b6ce822 100644 --- a/src/backend/commands/copyfromparse.c +++ b/src/backend/commands/copyfromparse.c @@ -1390,6 +1390,19 @@ CopyReadLineTextSIMDHelper(CopyFromState cstate, bool is_csv, /* Load more data if needed. */ if (copy_buf_len - input_buf_ptr < sizeof(Vector8)) { + /* + * In text mode, a backslash in the remaining bytes might start an + * end-of-copy marker. Check only for the backslash, leaving + * marker recognition and escape handling to the scalar code. Do + * this before reading more data, which could block on a pipe + * despite a complete marker being buffered. Otherwise, refill + * here so that long lines can continue to use SIMD. + */ + if (!is_csv && + memchr(copy_input_buf + input_buf_ptr, '\\', + copy_buf_len - input_buf_ptr) != NULL) + break; + REFILL_LINEBUF; CopyLoadInputBuf(cstate, true); -- 2.47.3
