Re: [Haskell-cafe] catting to cat gets stuck at > 135K
I am using 6.8.3 -- it is almost exciting to realize how close I came to 'impossible' instead of 'annoying'. Living on the edge with UNIX IO! -- _jsn ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] catting to cat gets stuck at > 135K
That was just me being absent-minded -- I have threaded in my Cabal file but was not using it on the command line for my little test script. Thank you for calling it to my attention. -- _jsn ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] catting to cat gets stuck at > 135K
"Jason Dusek" <[EMAIL PROTECTED]> writes: >> > simple exe bytes args= do >> > (i, o, e, p)<- runInteractiveProcess exe args Nothing >> > Nothing >> > hPut i bytes >> > s <- hGetContents o >> > hClose i >> > return s >> >> Yep, that's your problem. forkIO the hPut. > > Maybe I didn't do enough here -- just wrapping in `forkIO` > does not seem to actually help. What GHC version are you using? Bugs #1780, #1936, and #2155 might be relevant. -- Daniel Franke [EMAIL PROTECTED] http://www.dfranke.us || =|\ || * | -|-\- Man is free at the instant he wants to be. -| =| \ /// --Voltaire ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] catting to cat gets stuck at > 135K
On 2008 Nov 10, at 19:04, Jason Dusek wrote: simple exe bytes args= do (i, o, e, p)<- runInteractiveProcess exe args Nothing Nothing hPut i bytes s <- hGetContents o hClose i return s Yep, that's your problem. forkIO the hPut. Maybe I didn't do enough here -- just wrapping in `forkIO` does not seem to actually help. *sigh* I hate the ghc runtime... it works in ghci, or compiled with - threaded. Otherwise you still get the deadlock because it only switches threads under limited circumstances (garbage collections?) which isn't nearly often enough. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] [EMAIL PROTECTED] system administrator [openafs,heimdal,too many hats] [EMAIL PROTECTED] electrical and computer engineering, carnegie mellon universityKF8NH ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] catting to cat gets stuck at > 135K
This does not work either. It should cover all the bases, right? Fork off input, pull things from ouput as they are ready, stop when we reach end of file. If you remove the line `print partial`, the program loops forever; if you keep it, the program stops right there. -- _jsn import Data.ByteString.Lazy hiding (putStrLn) import System.Process import System.Environment import System.IO (putStrLn, hClose, hWaitForInput) import System.IO.Error import Control.Concurrent import Prelude hiding (writeFile, readFile) main = do exe:file:_<- getArgs bytes <- readFile file foo <- simple exe bytes [] writeFile (file ++ ".foo") foo -- Manufactures a simple stream handler from a command line utility. simple :: String -> ByteString -> [String] -> IO ByteString simple exe bytes args= do (i, o, e, p) <- runInteractiveProcess exe args Nothing Nothing pushAndPull i o bytes pushAndPull i o bytes= do putStrLn "Working with:" print bytes forkIO $ hPut i bytes putStrLn "forked" readUntilDone empty where readUntilDone soFar= do (const $ return soFar) `hctac` do putStrLn "hctac" hWaitForInput o 0 -- Wait as long as it takes. putStrLn "waited" partial <- hGetContents o putStrLn "contents:" print partial readUntilDone $ append soFar partial where hctac= flip catch ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] catting to cat gets stuck at > 135K
> > simple exe bytes args= do > > (i, o, e, p)<- runInteractiveProcess exe args Nothing > > Nothing > > hPut i bytes > > s <- hGetContents o > > hClose i > > return s > > Yep, that's your problem. forkIO the hPut. Maybe I didn't do enough here -- just wrapping in `forkIO` does not seem to actually help. -- _jsn import Data.ByteString.Lazy import System.Process import System.Environment import System.IO (hClose) import Control.Concurrent import Prelude hiding (writeFile, readFile) main = do exe:file:_<- getArgs bytes <- readFile file foo <- simple exe bytes [] writeFile (file ++ ".foo") foo -- Manufactures a simple stream handler from a command line utility. simple :: String -> ByteString -> [String] -> IO ByteString simple exe bytes args= do (i, o, e, p) <- runInteractiveProcess exe args Nothing Nothing forkIO $ hPut i bytes s <- hGetContents o return s ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] catting to cat gets stuck at > 135K
> > Yep, that's your problem. forkIO the hPut. > I can see the that thing biting a lot of people. Maybe there should be a warning in docs that this particular combination: > feed input > read output > waitForProcess is just likely to produce deadlocks? All best Christopher Skrzętnicki ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] catting to cat gets stuck at > 135K
On 2008 Nov 10, at 16:29, Jason Dusek wrote: I've put together a simple test case for a rather annoying problem. I've got a program that drives other programs. For example, it can drive `cat`: :; Simple cat a-file When the file is a little bit greater than 135060 bytes, this program fails to produce any output at all -- I need to use ^C to get my console back. If you are feeding it input and collecting output, you need to forkIO one or both. In particular, the sequence feed input read output waitForProcess will deadlock if at any point the input or output pipe fills: one side will be blocked on write() waiting for the other side to read() from the pipe, while the read() side is blocked waiting for write() on the other pipe, which won't be read() because that's the first side. (There are other variants of this, but that's the general form: processes blocked each waiting for the other side to do something.) If I remove the `hClose`, the example program just hangs, no matter the size of the input. That's another symptom of it, yes, made worse because the pipe close is now implicit and the other side won't stop read()ing until the pipe is close()d by the first side. simple exe bytes args= do (i, o, e, p)<- runInteractiveProcess exe args Nothing Nothing hPut i bytes s <- hGetContents o hClose i return s Yep, that's your problem. forkIO the hPut. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] [EMAIL PROTECTED] system administrator [openafs,heimdal,too many hats] [EMAIL PROTECTED] electrical and computer engineering, carnegie mellon universityKF8NH ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
