[
https://issues.apache.org/jira/browse/CASSANDRA-21381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18084207#comment-18084207
]
Brad Schoening edited comment on CASSANDRA-21381 at 5/28/26 8:55 PM:
---------------------------------------------------------------------
[~arvindk12] I don't see the benefit of supporting binary data (control
characters) in text values for CSV export. A few things give me pause:
* There are some real risks here around CSV injection, Trojan Source, other
forms of malicious data injection, and null-byte interactions with downstream
parsers.
* RFC 4180 explicitly excludes control characters from its grammar, and most
consumers (Excel, Java/Python CSV libraries, etc.) handle them inconsistently
or not at all.
* For round-tripping binary or control-char-bearing data, CSV doesn't feel like
the right format, a binary export format would handle this more cleanly.
The Jira doesn't explain why supporting control character is useful, especially
in light of the concerns above.
was (Author: bschoeni):
I don't see the benefit of supporting binary data (control characters) in text
values for CSV export. A few things give me pause:
* There are some real risks here around CSV injection, Trojan Source, other
forms of malicious data injection, and null-byte interactions with downstream
parsers.
* RFC 4180 explicitly excludes control characters from its grammar, and most
consumers (Excel, Java/Python CSV libraries, etc.) handle them inconsistently
or not at all.
* For round-tripping binary or control-char-bearing data, CSV doesn't feel like
the right format, a binary export format would handle this more cleanly.
The Jira doesn't explain why supporting control character is useful, especially
in light of the concerns above.
> CSV COPY TO corrupts control characters (newline, null byte, etc.) in text
> values
> ---------------------------------------------------------------------------------
>
> Key: CASSANDRA-21381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21381
> Project: Apache Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
> Reporter: Jens Geyer
> Assignee: Arvind Kandpal
> Priority: Normal
> Time Spent: 2h
> Remaining Estimate: 0h
>
> h2. Problem
> During COPY TO, control characters in text column values are replaced with
> their Python repr() notation by
> UNICODE_CONTROLCHARS_RE.sub(_show_control_chars, ...) in
> {{format_value_text}} ({{pylib/cqlshlib/formatting.py}}).
> Examples:
> * A stored newline (0x0A) becomes the two-character sequence {{\n}} in the
> CSV; after COPY FROM it is stored as {{\n}} (backslash + n) -- a different
> value.
> * A null byte (0x00) becomes {{\x00}} (six characters).
> The regex {{UNICODE_CONTROLCHARS_RE = re.compile(r"[\x00-\x1f\x7f-\xa0]")}}
> matches all ASCII control characters (0x00-0x1F: newline, tab, carriage
> return, BEL, etc.) and Latin-1 supplement control characters (0x7F-0xA0).
> This substitution is correct for terminal display of SELECT results (where
> invisible characters need a human-readable representation). It is incorrect
> in the *CSV export path*, where {{csv.writer}} handles control characters
> correctly via field quoting -- no pre-processing is needed.
> h2. Affected Versions
> All Cassandra versions with {{format_value_text}} containing the
> {{UNICODE_CONTROLCHARS_RE}} substitution (at minimum 3.x through trunk).
> h2. Root Cause
> {{format_value_text}} is shared between the terminal display path (SELECT
> output) and the CSV export path (COPY TO). The {{UNICODE_CONTROLCHARS_RE}}
> substitution converts control characters to their Python repr-string for
> display, but this transformation is *not reversible* via the CSV import path.
> This bug is *independent of, but in the same function as*, the
> backslash-doubling bug fixed in CASSANDRA-21131. Applying the CASSANDRA-21131
> patch does NOT fix this issue.
> h2. Expected Fix
> In the CSV export path, skip the {{UNICODE_CONTROLCHARS_RE.sub(...)}} call.
> An {{escape_control_chars}} parameter (analogous to the {{escape_backslash}}
> parameter introduced by CASSANDRA-21131) can conditionally suppress the
> substitution when calling {{format_value_text}} from the CSV export path.
> h2. Related
> CASSANDRA-21131 -- backslash-doubling bug in the same code path, already
> patched.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
