A NOTE has been added to this issue. ====================================================================== https://austingroupbugs.net/view.php?id=1927 ====================================================================== Reported By: dwheeler Assigned To: ajosey ====================================================================== Project: 1003.1(2008)/Issue 7 Issue ID: 1927 Category: Shell and Utilities Type: Clarification Requested Severity: Editorial Priority: normal Status: Under Review Name: David A. Wheeler Organization: The Linux Foundation User Reference: Utilities Section: Utilities Page Number: NA Line Number: NA Interp Status: --- Final Accepted Text: ====================================================================== Date Submitted: 2025-06-01 01:18 UTC Last Modified: 2025-06-22 08:43 UTC ====================================================================== Summary: Add sponge utility ======================================================================
---------------------------------------------------------------------- (0007207) stephane (reporter) - 2025-06-22 08:43 https://austingroupbugs.net/view.php?id=1927#c7207 ---------------------------------------------------------------------- Sponge is part of a collection of utilities that started out as mostly a bunch of useful perl scripts. Initially, sponge file was in essence: perl -0777 -pe 'open STDOUT, ">", file' Soaking up the input in memory, and writing it to "file" upon eof on stdin, so behaving more like POSIX ed, ex, sort -o. At some point, it was rewritten in C and changed its modus operandi radically. It behaved more like perl -i / gsed -i, which replaces the file with an edited copy, except that: - it uses a tempfile in ${TMPDIR-/tmp} instead of the same directory of the file which makes it much less likely for rename() to succeed as /tmp is often on a separate FS - Like perl, unlike gsed, it only tries to preserve basic Unix permissions, not extended ACLs, ownership or other attributes such as security context. - If the rename fails, it's back to ex/ed behaviour where it copies the temp file to the original file à la ed/ex while perl/gsed -i would fail (unlikely in practice there when the tempfile is in the same directory as the original) Both approaches (overwrite existing file vs replace it with a new one) each have their advantage / drawbacks: Overwriting means: - inode number, permissions, ownership and most metadata is preserved. - but is not atomic - can't be done for files that are in use executables for instance and can cause havoc for scripts that are currently being run. Replacing means: - it's atomic and generally avoids the problems above. - does not preserve inode number, cannot always preserve metadata (such as ownership or some security attributes that require privileges to set) - breaks hard links - breaks symlinks (replaces them with a regular file, losing metadata (the target of the symlink) in the process. Newer sponge's hybrid approach means it's still able to change a file in a directory the user doesn't have write access to as long as the user has write access to the file, but then that means non-atomic replace and different behaviour for links. IOW, sponge can end up doing very different things, not based on what the user decides but based on external factors, which IMO is not a clean design. If sponge was to be specified, I'd rather it was the original behaviour (which reflects the "sponge" name better) or the gsed -i one. I don't know of a system that has that utility installed by default and personally haven't felt I couldn't do without it. I'm not fully convinced it's worth specifying. It's nowhere near the bulletproof way to edit files in place that one may hope it to be. One problem is that in: cmd < file | sponge file file is overwritten and lost even if cmd fails or file can't be opened for reading. { cmd | sponge file; } < file Addresses one of the problems. I'd say process substitution would be a feature more worth specifying than a sponge utility. In particular zsh's =(...) form that uses a temp file can replace most usages of sponge (though comes with its own limitations) as in: cp =(cmd file) file To replace: cmd file | sponge file { rm -f file && cmd > file; } < file Can also already replace many usages of sponge (and to some extent is safer than < file cmd | sponge file). See also ksh93's: grep foo < file 1<>; file To overwrite the file in place (valid for things like "grep" that don't produce more output than they consume) and truncates in the end. Issue History Date Modified Username Field Change ====================================================================== 2025-06-01 01:18 dwheeler New Issue 2025-06-01 01:18 dwheeler Status New => Under Review 2025-06-01 01:18 dwheeler Assigned To => ajosey 2025-06-11 16:49 dwheeler Note Added: 0007198 2025-06-22 08:43 stephane Note Added: 0007207 ======================================================================
