Re: [dev] situ — change files in situ
On 13/04/2012, Truls Becken wrote: > You are right. I thought that since it only opens the output file > after stdin is fully consumed, it would not touch the file on failure. > The test below reveals that sponge leaves the file alone if the > process is interrupted before end of stdin, but not if a program > before it in the pipe returns non-zero. > > I guess I should test programs before suggesting them to others. (^_^) Actually, it's not wrong — it would do the job, i.e. to mutate file in situ — it's just not quite sure enough. Cheers, strake
Re: [dev] situ — change files in situ
On Fri, Apr 13, 2012 at 19:50, Strake wrote: >> There is also sponge from http://kitenet.net/~joey/code/moreutils/ > > which would also corrupt file on failure, by what little documentation > I can read through the markup. You are right. I thought that since it only opens the output file after stdin is fully consumed, it would not touch the file on failure. The test below reveals that sponge leaves the file alone if the process is interrupted before end of stdin, but not if a program before it in the pipe returns non-zero. I guess I should test programs before suggesting them to others. -Truls $ echo TEST > tst $ (sed s/ES/se tst; sleep 10) | sponge tst sed: -e expression #1, char 7: unterminated `s' command ^C $ cat tst TEST $ sed s/ES/se tst | sponge tst sed: -e expression #1, char 7: unterminated `s' command $ wc tst 0 0 0 tst $
Re: [dev] situ — change files in situ
On 13/04/2012, Truls Becken wrote: > There is also sponge from http://kitenet.net/~joey/code/moreutils/ which would also corrupt file on failure, by what little documentation I can read through the markup.
Re: [dev] situ — change files in situ
There is also sponge from http://kitenet.net/~joey/code/moreutils/ -Truls
Re: [dev] situ — change files in situ
On 13/04/2012, Connor Lane Smith wrote: > On 13 April 2012 16:37, Strake wrote: >> True, if we trim the start and finish lazily > > That depends on the diff format. diff -e, for example, could work well. I meant to trim pre-diff, so it would be quadratic in a lesser argument. >> but worst-case diff >> algorithm space usage is worse — quadratic, if I'm not mistaken. > > That's true. Although, diff is worst-case quadratic in the number of > lines, not bytes. True for line-by-line diff, e.g. Unix diff, though the algorithm is more general. > And the fact is the worst-case can be avoided: we > don't really *need* the longest common subsequence, only an > approximation to it. If we exceed some threshold we can simply > 'plump'. You mean, to terminate the subsequence, and start afresh? > To be clear, I don't mean to dismiss either existing situ, this would > just be nice to have as well. Noted. Cheers, strake
Re: [dev] situ — change files in situ
On 13 April 2012 16:37, Strake wrote: > True, if we trim the start and finish lazily That depends on the diff format. diff -e, for example, could work well. > but worst-case diff > algorithm space usage is worse — quadratic, if I'm not mistaken. That's true. Although, diff is worst-case quadratic in the number of lines, not bytes. And the fact is the worst-case can be avoided: we don't really *need* the longest common subsequence, only an approximation to it. If we exceed some threshold we can simply 'plump'. To be clear, I don't mean to dismiss either existing situ, this would just be nice to have as well. cls
Re: [dev] situ — change files in situ
On 13/04/2012, Connor Lane Smith wrote: > Your method, while simpler, requires 2n space to 'situ' a length n > file. A suitable patch format would require only enough space to store > the changes to the original file. The outcome would be identical, but > if we're trying to sed only a few changes in a large file it would be > more efficient in terms of storage. True, if we trim the start and finish lazily, but worst-case diff algorithm space usage is worse — quadratic, if I'm not mistaken.
Re: [dev] situ — change files in situ
On 13 April 2012 16:04, Strake wrote: > I'm not sure why — this method seems ultimately functionally equivalent to > mine. > Perhaps I fail to ken your logic here. Your method, while simpler, requires 2n space to 'situ' a length n file. A suitable patch format would require only enough space to store the changes to the original file. The outcome would be identical, but if we're trying to sed only a few changes in a large file it would be more efficient in terms of storage. cls
Re: [dev] situ — change files in situ
On 13/04/2012, Connor Lane Smith wrote: > I wonder whether a situ-like could be used to produce a diff of the > original file, rather than a complete second copy. Programs like sed > don't generally make that many changes, so if we were to connect situ > to the file and the cmd's stdout we could compute the diff and then > patch the original file once cmd exits 0. If someone could get that > working it would be pretty sweet. I'm not sure why — this method seems ultimately functionally equivalent to mine. Perhaps I fail to ken your logic here. Cheers, strake
Re: [dev] situ — change files in situ
On 13/04/2012, Troels Henriksen wrote: > I wrote a similar program in C: http://sigkill.dk/projects/insitu/ It > also does actual in-replace replacement (which is sometimes not what you > want). I actually found this prior, but wanted a program that would not harm the file on failure. > Note that you can also exploit shell evaluation order: What cls said.
Re: [dev] situ — change files in situ
Hey, On 13 April 2012 14:14, Troels Henriksen wrote: > Note that you can also exploit shell evaluation order Of course, that doesn't wait to see whether cmd succeeds. I wonder whether a situ-like could be used to produce a diff of the original file, rather than a complete second copy. Programs like sed don't generally make that many changes, so if we were to connect situ to the file and the cmd's stdout we could compute the diff and then patch the original file once cmd exits 0. If someone could get that working it would be pretty sweet. Thanks, cls
Re: [dev] situ — change files in situ
Strake writes: > Hello all. > > I wrote a program to mutate files in situ by a given shell program, > available here: > http://strake.zanity.net:1104/situ/ > > I thought that it might be useful for some other folks here, ergo this mail. > I wrote it to gain the functionality of GNU sed's -i switch in 9base sed. > Unlike GNU sed's -i switch, it ought to work with any program. I wrote a similar program in C: http://sigkill.dk/projects/insitu/ It also does actual in-replace replacement (which is sometimes not what you want). Note that you can also exploit shell evaluation order: (rm foo && cmd... > foo) < foo This will do an (almost) in-place replacement of foo. -- \ Troels /\ Henriksen
[dev] situ — change files in situ
Hello all. I wrote a program to mutate files in situ by a given shell program, available here: http://strake.zanity.net:1104/situ/ I thought that it might be useful for some other folks here, ergo this mail. I wrote it to gain the functionality of GNU sed's -i switch in 9base sed. Unlike GNU sed's -i switch, it ought to work with any program. Cheers, strake
