Dear coreutils team,
We found a bug in csplit that loses data when the input file has the
name of one of the output files (xx00, xx01, ... by default). csplit
creates (and so truncates) each output file before reading the part of
the input that goes into it. If the input is one of these files and has
not yet been read to the end, the part not yet read is lost.
For xx00 this is all of the input, since nothing has been read yet. For
the later output files, how far csplit has read is the previous split
point rounded up to its read buffer, so whether data is lost depends on
where the input is split, not only on its size. After the truncation,
csplit sees an early end of file, and usually exits with status 0. If a
later pattern then fails, csplit removes the output files it has
created, the input among them.
With regular expressions, both how far csplit has read and which output
files are created depend on the input's contents: the split points are
where the expressions match, and with a repeat count of {*} the number
of output files is the number of matches. A leading %REGEX% only delays
the creation of xx00 until the skipped lines have been read.
We reproduced this with versions 8.32 (Ubuntu 22.04.5 LTS), 9.4 (Ubuntu
24.04) and 9.7 (Debian 13). The results below are the same in all three
except where noted; byte counts are from 9.4 and 9.7 (8.32 differs by
one byte).
This bug was found by João Matos (joaorsmatos02) and Frederico Ramos
(frediramos), as part of a research project at Instituto Superior
Técnico, University of Lisbon (https://tecnico.ulisboa.pt/) and Carnegie
Mellon University (https://www.cmu.edu/).
## Environment
- Coreutils version: GNU coreutils 9.7
- Operating system: Debian GNU/Linux 13
- Architecture: x86_64
- Locale: C.UTF8 (LANG=C.UTF-8, LC_ALL unset)
## Steps to reproduce
### Case 1: the input is named xx00
$ seq 10 > xx00 && csplit xx00 5
8.32 and 9.4: "csplit: input disappeared", exit status 1, xx00 is left
empty. 9.7: "csplit: '5': line number out of range", exit status 1, xx00
is deleted. Where the input is split does not matter.
### Case 2: a later output file, silent loss
$ seq 300000 > xx01 && csplit xx01 2
2
8188
Exit status 0, no message. xx01 now holds 8188 bytes; the input had 1988895.
### Case 3: a later pattern fails, and the input is deleted
$ seq 300000 > xx01 && csplit xx01 2 50000
csplit: '50000': line number out of range
8188
Exit status 1. Both xx00 and xx01 are removed: nothing is left of the
input. The same happens with regular expressions:
$ seq 300000 > xx01 && csplit xx01 '/^1000/' '/299990/'
csplit: '/^299990$/': match not found
3888
csplit: '/299990/': match not found
4302
### Case 4: {*}: which files are at risk depends on the contents
$ seq 300000 > xx02 && csplit xx02 '/0000$/' '{*}'
Exit status 0. Only xx00 to xx02 are created, and xx02 holds 5766 bytes.
The same command on a file named xx31 loses nothing: there are 30
matches, so only xx00 to xx30 are created.
### Case 5: a leading %REGEX%
$ seq 300000 > xx00 && csplit xx00 '%^5$%' 10
10
8172
Exit status 0. xx00 holds lines 5 to 9 as expected, but xx01, which
should hold the rest of the input, holds only 8172 bytes.
## Fix
We have a patch (along with a new test) that does what split already
does: opens each output file without O_TRUNC, compares it with the input
(same device and inode), and fails with "'xx00' would overwrite input;
aborting" before writing anything. The input is left intact.
Should we do it publicly in order to be properly credited? Or would you
prefer to open a private branch for us on Github?
Do not hesitate to contact us for any further details.
Best regards,
João and Frederico
