On 17/03/2021 23:12, Chet Ramey wrote:
I didn't expect it to be perfect (see thx etc below). Better: I needed help in telling when it would most like fail! :)On 3/17/21 3:29 PM, Michael Felt wrote:I tried as many combinations of commands as I could - and it seems that the regular behavior of dup2 on the opened fifo is enough to maintain communication.It's not, since FIFOs exist in the file system and have to be available toopen(2) when the other process (consumer, producer) wants to use them.
Going into a system test (ie. a normal AdoptOpemJDK build process) that has nearly 3500 commands, each with two process_substitution commands.Consider the following scenario. You want to perform a regression test of two versions of a program, each of which produces textual output. You run diff <(program-version-1 args) <(program-version-2 args)
Yes, something to test. Thx. The ojdk scenario is: /usr/bin/printf > >(tee -a stdout.log) 2> >(tee -a stderr.log).
So, yes, in this case it is working because printf is the parent - (which I never seemed to find actually calling open() of the file. It seems to be using the fd opened by the child - in a magical way).
But what I thoght I was seeing is that diff is the PARENT calling substitute_process() that create(s) a child process that reads/writes to a fifo file. a) the child process never returns - it `exits` via, iirc, sh_exit(result) and the end of the routine b) the parent gets the filename (pathname) - but I never see it actually opening it - only (when using bash -x) seeing the name in the -x command expansion.This is defined to provide `diff' with two arguments. Let's call them /var/tmp/sh-np12345 and /var/tmp/sh-np67890 So diff runs, sees two arguments, opens both files, and does its thing. Diff has to see two filenames when it runs, otherwise it's an error.
No I cannot - and for now it is a `hack` to solve a bigger issue. With 3500 calls in a single build I hope the race occurs - and I'll finally see where the PARENT actually uses the name returned.Now, let's say your change is there. The shell still runs diff /var/tmp/sh-np12345 /var/tmp/sh-np67890 but, depending on how processes get scheduled, the shell forked to run the process substitutions has already unlinked those FIFOs. Diff will error out. The user will be unhappy. I will get bug reports. You have introduced a race condition. You may not get hit by it, but you cannot guarantee that no one will.
Again, my issue was with >(command) substitution - where the `files` get written to by the parent - rather than reading them.The shell can't unlink the FIFO until it can guarantee that the processes that need to open it have opened it, and it can't guarantee that in the general case. It has to wait until the process completes, at least, and even that might not be correct.
p.s. it is not my call to ask why they do not use regular redirection or pipes. Feels much simpler - but some people cannot miss the opportunity to use something blinky and shiney.
p.p.s. - If you have `words of wisdom` re: why this approach is much better than `standard` redirection - I am all ears!
*** Thanks again for the time to reply ***
That's why the last-ditch approach is to remove all remaining FIFOs when the shell exits.btw: other than the one open in the middle of process_substitution() I did not see anywhere where another process even tries to open the file.They are not necessarily shell processes, but what if they were? Since a FIFO is an object in the file system, you can just open(2) it. That's ostensibly the advantage of FIFOs.what I also noticed is that the process, (iirc) that opens the file - never 'returns' - it ends via sh_exit() and the end of the routine.Of course. It's a process that is forked to run the command specified in the process substitution. What else does it need to do?
OpenPGP_0x722BFDB61F396FC2.asc
Description: OpenPGP public key
OpenPGP_signature
Description: OpenPGP digital signature