Dear DMTCP team -- I have some indirect evidence of a possible problem around pipes.
I have a variety of jobs that take the form of gunzip to a pipe -> some Java program -> pipe to gzip The pipes are sometimes done as actual pipes using | in bash. In some cases I have more than one output file that needs to be gzipped, so I had been using this bash incantation: java ... >(gzip ....) This creates a pipe on some fd, say 253, on the Java end, and has it be standard input to the gzip. The >(...) stuff is then replaced on the Java command line by /dev/fd253, i.e., Java is presented with a filename that it can open. This seems to fail in DMTCP, presumably because it plays around with fd's and pipes. So my workaround is to develop a filename, say foo.pipe, and do: rm -f foo.pipe mkfifo foo.pipe java ... -out foo.pipe & gzip < foo.pipe > foo.gz rm -f foo.pipe This works as one would hope. EXCEPT! Sometimes when running under DMTCP some pipe data seems to be lost, so foo.gz does not contain all the data it is supposed to. Urg! I have sometimes seen DMTCP warning about taking too long to drain a socket (or some such). Perhaps it is related? I am not sure if it is specific to Java, has to do with (ordinary) pipes, named pipes, or what, but the problem goes away when I run without DMTCP. So again, I am in a situation where I cannot use the tool and have to hope my jobs complete without an intervening power failure ... Regards -- Eliot Moss ------------------------------------------------------------------------------ _______________________________________________ Dmtcp-forum mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dmtcp-forum
