A NOTE has been added to this issue. ====================================================================== https://www.austingroupbugs.net/view.php?id=1927 ====================================================================== Reported By: dwheeler Assigned To: ajosey ====================================================================== Project: 1003.1(2008)/Issue 7 Issue ID: 1927 Category: Shell and Utilities Type: Clarification Requested Severity: Editorial Priority: normal Status: Under Review Name: David A. Wheeler Organization: The Linux Foundation User Reference: Utilities Section: Utilities Page Number: NA Line Number: NA Interp Status: --- Final Accepted Text: ====================================================================== Date Submitted: 2025-06-01 01:18 UTC Last Modified: 2025-07-04 02:11 UTC ====================================================================== Summary: Add sponge utility ======================================================================
---------------------------------------------------------------------- (0007219) dwheeler (reporter) - 2025-07-04 02:11 https://www.austingroupbugs.net/view.php?id=1927#c7219 ---------------------------------------------------------------------- Obviously there are various ways to implement a "sponge-like" functionality. As noted above, the original sponge developers originally overwrote the file. However, years ago they changed to a hybrid "do the best you can" approach that prefers an ACID overwrite where it can manage it, and an overwrite when it can't. The fact that they *changed* the code to do this, even though it takes more code, suggests that this is the most widely expected default ("do the best you can"). "Doing what most people want by default" is a *good* design. The fact that there *are* trade-offs suggests that perhaps options could be added control what approach is used, in cases where users care. (E.g., only overwrite, only replace, and prefer-replace-then-overwrite). It seems like this is exactly what options are good for :-). Regarding "it uses a tempfile in ${TMPDIR-/tmp} instead of the same directory of the file which makes it much less likely for rename() to succeed as /tmp is often on a separate FS", you could set its TMPDIR to be the same directory as the destination file. If that's a common case, an option could be added to make that even easier to do. "Like perl, unlike gsed, it only tries to preserve basic Unix permissions, not extended ACLs, ownership or other attributes such as security context." It's sometimes not even *possible* to retain security context (even in an overwrite!). However, I can see the specification encouraging implementations to maximally preserve permissions, at *least* basic Unix permission bits. Then it's a quality-of-implementation issue. "[In] cmd < file | sponge file file is overwritten and lost even if cmd fails or file can't be opened for reading.... { cmd | sponge file; } < file" I don't see that as a serious problem. In-place replacement is typically *NEVER* used on precious irreplacement initial inputs. It's typically used as part of a larger sequence of pipelines with potentially large files. I'll agree that if that *is* a concern of a user, it's challenging to handle. I guess it could grow an option to only replace a file if it got at least 1 byte of input, and return an error code if it didn't. I don't have any other ideas if that's critical, but again, normally you only do in-place replacements when it's replaceable. Process substitution is different from sponge. Constructs like "cp =(cmd file) file" don't preserve ANY permissions, for example. I wouldn't oppose process substitution in a different proposal, but that seems different. David Leonard: "My first thought is that sponge may benefit from a flock option...." I'm quite amenable to that! I don't know if the developers of the most widely-used implementation of sponge would be willing to add a few new options, but I think it's worth asking. All of these additions (even working harder to copy the permission information) aren't really *that* much work on top of what's already there. Even with all those options, it's a narrowly-scoped utility that isn't hard to use or implement. Issue History Date Modified Username Field Change ====================================================================== 2025-06-01 01:18 dwheeler New Issue 2025-06-01 01:18 dwheeler Status New => Under Review 2025-06-01 01:18 dwheeler Assigned To => ajosey 2025-06-11 16:49 dwheeler Note Added: 0007198 2025-06-22 08:43 stephane Note Added: 0007207 2025-06-26 15:28 eblake Note Added: 0007211 2025-06-26 15:41 eblake Note Edited: 0007211 2025-06-26 15:45 eblake Note Added: 0007212 2025-06-26 17:39 enh Note Added: 0007214 2025-07-04 02:11 dwheeler Note Added: 0007219 ======================================================================
