Re: [Haskell-cafe] Conduit Error Output: Control.Monad.Trans.Resource.stateCleanup
That makes a lot of sense. The diff was a bit beyond my skimming abilities :-) On Tue, Feb 21, 2012 at 4:15 PM, Michael Snoyman wrote: > Hi Lyndon, > > Outputting nothing *is* the desired result. When you run something like: > > bar <- runResourceT $ lazyConsume $ sourceFile "foo.txt" > print bar > > The steps that occur are roughly[1]: > > * ResourceT is initialized > * File handle is opened, release action is registered to close the file handle > * unsafeInterleaveIO is called, which creates a thunk that will pull > from the Handle > * runResourceT is called, which calls all release actions, including > closing the file handle > * The thunk is evaluated. It checks if the ResourceT is open. Since it > isn't, it returns a []. > > The problem previously is that the last step was not checking if the > ResourceT was still open, which could result in pulling from a closed > handle. > > Michael > > On Tue, Feb 21, 2012 at 9:57 AM, Lyndon Maydwell wrote: >> Hi Michael, >> >> >> The behaviour of my original code has now changed to output nothing >> with no errors. I'm not sure of the significance of this as my code >> was incorrect, however, using the code you demonstrated gives the >> desired results. >> >> Thanks for the blindingly quick response! >> >> >> On Tue, Feb 21, 2012 at 3:30 PM, Michael Snoyman wrote: >>> On Tue, Feb 21, 2012 at 7:40 AM, Michael Snoyman >>> wrote: On Tue, Feb 21, 2012 at 5:46 AM, Lyndon Maydwell wrote: > Hi Michael, Café. > > I'm writing some code using the conduit library and am encountering > the following error output (while the program appears to function > correctly) when using Data.Conduit.Lazy. > > The error given is: > >> profile_simple_test_data: Control.Monad.Trans.Resource.stateCleanup: >> There is a bug in the implementation. The mutable state is being >> accessed after cleanup. Please contact the maintainers. > > A reduced code snippet that generates this error is (also attached): > >> import Control.Monad >> import System.Environment >> import Control.Monad.IO.Class (liftIO) >> import System.IO >> import Data.Conduit.Lazy >> import Data.List (sort) >> >> import Data.Conduit >> >> import Prelude hiding (map) >> >> main = getArgs >>= process >> >> process args = mapM_ sorted args >> >> sorted x = runResourceT (lazyConsume $ sourceFeed x) >>= (mapM_ print . >> id) >> >> sourceFeed :: ResourceIO m => FilePath -> Source m String >> sourceFeed file = sourceIO >> (openFile file ReadMode) >> hClose >> (\h -> liftIO $ do >> eof <- hIsEOF h >> if eof >> then return IOClosed >> else fmap IOOpen $ hGetLine h) > > when run over any text file. > > I may be doing something inconsistent with the correct use of sourceIO > or lazyConsume, however, I tried to follow the example at > http://www.yesodweb.com/home/snoyberg/blogs/conduit/conduit/source/source.ditamap?nav=nav-2 > as closely as possible. > > Is this a bug, or simply an incorrect use of Conduit? I haven't fully debugged this yet. There's certainly a bug in the implementation of ResourceT, but the sample program is also wrong. You can't pass the result from a call to lazyConsume outside the scope of its ResourceT; the correct way to write sorted would be: sorted x = runResourceT $ lazyConsume (sourceFeed x) >>= mapM_ (liftIO . print) My guess is that this is a fallout from the transition away from mutable variables: lazyConsume no longer has any way of knowing that its ResourceT has already been terminated. Perhaps a simple solution would be to expose a primitive that checks if the ResourceT block has already been finalized. Michael >>> >>> I've added a test case for this bug, and fixed it. The commit is: >>> >>> https://github.com/snoyberg/conduit/commit/87e890fe7ee58686d20cabba15dd37f18ba66620 >>> >>> The basic idea is to add an extra constructor to represent when the >>> ResourceT has already been closed, and expose a function >>> resourceActive to check the state. Can you check if this solves your >>> problem? >>> >>> Michael ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Conduit Error Output: Control.Monad.Trans.Resource.stateCleanup
Hi Lyndon, Outputting nothing *is* the desired result. When you run something like: bar <- runResourceT $ lazyConsume $ sourceFile "foo.txt" print bar The steps that occur are roughly[1]: * ResourceT is initialized * File handle is opened, release action is registered to close the file handle * unsafeInterleaveIO is called, which creates a thunk that will pull from the Handle * runResourceT is called, which calls all release actions, including closing the file handle * The thunk is evaluated. It checks if the ResourceT is open. Since it isn't, it returns a []. The problem previously is that the last step was not checking if the ResourceT was still open, which could result in pulling from a closed handle. Michael On Tue, Feb 21, 2012 at 9:57 AM, Lyndon Maydwell wrote: > Hi Michael, > > > The behaviour of my original code has now changed to output nothing > with no errors. I'm not sure of the significance of this as my code > was incorrect, however, using the code you demonstrated gives the > desired results. > > Thanks for the blindingly quick response! > > > On Tue, Feb 21, 2012 at 3:30 PM, Michael Snoyman wrote: >> On Tue, Feb 21, 2012 at 7:40 AM, Michael Snoyman wrote: >>> On Tue, Feb 21, 2012 at 5:46 AM, Lyndon Maydwell wrote: Hi Michael, Café. I'm writing some code using the conduit library and am encountering the following error output (while the program appears to function correctly) when using Data.Conduit.Lazy. The error given is: > profile_simple_test_data: Control.Monad.Trans.Resource.stateCleanup: > There is a bug in the implementation. The mutable state is being accessed > after cleanup. Please contact the maintainers. A reduced code snippet that generates this error is (also attached): > import Control.Monad > import System.Environment > import Control.Monad.IO.Class (liftIO) > import System.IO > import Data.Conduit.Lazy > import Data.List (sort) > > import Data.Conduit > > import Prelude hiding (map) > > main = getArgs >>= process > > process args = mapM_ sorted args > > sorted x = runResourceT (lazyConsume $ sourceFeed x) >>= (mapM_ print . > id) > > sourceFeed :: ResourceIO m => FilePath -> Source m String > sourceFeed file = sourceIO > (openFile file ReadMode) > hClose > (\h -> liftIO $ do > eof <- hIsEOF h > if eof > then return IOClosed > else fmap IOOpen $ hGetLine h) when run over any text file. I may be doing something inconsistent with the correct use of sourceIO or lazyConsume, however, I tried to follow the example at http://www.yesodweb.com/home/snoyberg/blogs/conduit/conduit/source/source.ditamap?nav=nav-2 as closely as possible. Is this a bug, or simply an incorrect use of Conduit? >>> >>> I haven't fully debugged this yet. There's certainly a bug in the >>> implementation of ResourceT, but the sample program is also wrong. You >>> can't pass the result from a call to lazyConsume outside the scope of >>> its ResourceT; the correct way to write sorted would be: >>> >>> sorted x = runResourceT $ lazyConsume (sourceFeed x) >>= mapM_ >>> (liftIO . print) >>> >>> My guess is that this is a fallout from the transition away from >>> mutable variables: lazyConsume no longer has any way of knowing that >>> its ResourceT has already been terminated. Perhaps a simple solution >>> would be to expose a primitive that checks if the ResourceT block has >>> already been finalized. >>> >>> Michael >> >> I've added a test case for this bug, and fixed it. The commit is: >> >> https://github.com/snoyberg/conduit/commit/87e890fe7ee58686d20cabba15dd37f18ba66620 >> >> The basic idea is to add an extra constructor to represent when the >> ResourceT has already been closed, and expose a function >> resourceActive to check the state. Can you check if this solves your >> problem? >> >> Michael ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Conduit Error Output: Control.Monad.Trans.Resource.stateCleanup
Hi Michael, The behaviour of my original code has now changed to output nothing with no errors. I'm not sure of the significance of this as my code was incorrect, however, using the code you demonstrated gives the desired results. Thanks for the blindingly quick response! On Tue, Feb 21, 2012 at 3:30 PM, Michael Snoyman wrote: > On Tue, Feb 21, 2012 at 7:40 AM, Michael Snoyman wrote: >> On Tue, Feb 21, 2012 at 5:46 AM, Lyndon Maydwell wrote: >>> Hi Michael, Café. >>> >>> I'm writing some code using the conduit library and am encountering >>> the following error output (while the program appears to function >>> correctly) when using Data.Conduit.Lazy. >>> >>> The error given is: >>> profile_simple_test_data: Control.Monad.Trans.Resource.stateCleanup: There is a bug in the implementation. The mutable state is being accessed after cleanup. Please contact the maintainers. >>> >>> A reduced code snippet that generates this error is (also attached): >>> import Control.Monad import System.Environment import Control.Monad.IO.Class (liftIO) import System.IO import Data.Conduit.Lazy import Data.List (sort) import Data.Conduit import Prelude hiding (map) main = getArgs >>= process process args = mapM_ sorted args sorted x = runResourceT (lazyConsume $ sourceFeed x) >>= (mapM_ print . id) sourceFeed :: ResourceIO m => FilePath -> Source m String sourceFeed file = sourceIO (openFile file ReadMode) hClose (\h -> liftIO $ do eof <- hIsEOF h if eof then return IOClosed else fmap IOOpen $ hGetLine h) >>> >>> when run over any text file. >>> >>> I may be doing something inconsistent with the correct use of sourceIO >>> or lazyConsume, however, I tried to follow the example at >>> http://www.yesodweb.com/home/snoyberg/blogs/conduit/conduit/source/source.ditamap?nav=nav-2 >>> as closely as possible. >>> >>> Is this a bug, or simply an incorrect use of Conduit? >> >> I haven't fully debugged this yet. There's certainly a bug in the >> implementation of ResourceT, but the sample program is also wrong. You >> can't pass the result from a call to lazyConsume outside the scope of >> its ResourceT; the correct way to write sorted would be: >> >> sorted x = runResourceT $ lazyConsume (sourceFeed x) >>= mapM_ >> (liftIO . print) >> >> My guess is that this is a fallout from the transition away from >> mutable variables: lazyConsume no longer has any way of knowing that >> its ResourceT has already been terminated. Perhaps a simple solution >> would be to expose a primitive that checks if the ResourceT block has >> already been finalized. >> >> Michael > > I've added a test case for this bug, and fixed it. The commit is: > > https://github.com/snoyberg/conduit/commit/87e890fe7ee58686d20cabba15dd37f18ba66620 > > The basic idea is to add an extra constructor to represent when the > ResourceT has already been closed, and expose a function > resourceActive to check the state. Can you check if this solves your > problem? > > Michael ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Conduit Error Output: Control.Monad.Trans.Resource.stateCleanup
On Tue, Feb 21, 2012 at 7:40 AM, Michael Snoyman wrote: > On Tue, Feb 21, 2012 at 5:46 AM, Lyndon Maydwell wrote: >> Hi Michael, Café. >> >> I'm writing some code using the conduit library and am encountering >> the following error output (while the program appears to function >> correctly) when using Data.Conduit.Lazy. >> >> The error given is: >> >>> profile_simple_test_data: Control.Monad.Trans.Resource.stateCleanup: There >>> is a bug in the implementation. The mutable state is being accessed after >>> cleanup. Please contact the maintainers. >> >> A reduced code snippet that generates this error is (also attached): >> >>> import Control.Monad >>> import System.Environment >>> import Control.Monad.IO.Class (liftIO) >>> import System.IO >>> import Data.Conduit.Lazy >>> import Data.List (sort) >>> >>> import Data.Conduit >>> >>> import Prelude hiding (map) >>> >>> main = getArgs >>= process >>> >>> process args = mapM_ sorted args >>> >>> sorted x = runResourceT (lazyConsume $ sourceFeed x) >>= (mapM_ print . id) >>> >>> sourceFeed :: ResourceIO m => FilePath -> Source m String >>> sourceFeed file = sourceIO >>> (openFile file ReadMode) >>> hClose >>> (\h -> liftIO $ do >>> eof <- hIsEOF h >>> if eof >>> then return IOClosed >>> else fmap IOOpen $ hGetLine h) >> >> when run over any text file. >> >> I may be doing something inconsistent with the correct use of sourceIO >> or lazyConsume, however, I tried to follow the example at >> http://www.yesodweb.com/home/snoyberg/blogs/conduit/conduit/source/source.ditamap?nav=nav-2 >> as closely as possible. >> >> Is this a bug, or simply an incorrect use of Conduit? > > I haven't fully debugged this yet. There's certainly a bug in the > implementation of ResourceT, but the sample program is also wrong. You > can't pass the result from a call to lazyConsume outside the scope of > its ResourceT; the correct way to write sorted would be: > > sorted x = runResourceT $ lazyConsume (sourceFeed x) >>= mapM_ > (liftIO . print) > > My guess is that this is a fallout from the transition away from > mutable variables: lazyConsume no longer has any way of knowing that > its ResourceT has already been terminated. Perhaps a simple solution > would be to expose a primitive that checks if the ResourceT block has > already been finalized. > > Michael I've added a test case for this bug, and fixed it. The commit is: https://github.com/snoyberg/conduit/commit/87e890fe7ee58686d20cabba15dd37f18ba66620 The basic idea is to add an extra constructor to represent when the ResourceT has already been closed, and expose a function resourceActive to check the state. Can you check if this solves your problem? Michael ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Conduit Error Output: Control.Monad.Trans.Resource.stateCleanup
On Tue, Feb 21, 2012 at 5:46 AM, Lyndon Maydwell wrote: > Hi Michael, Café. > > I'm writing some code using the conduit library and am encountering > the following error output (while the program appears to function > correctly) when using Data.Conduit.Lazy. > > The error given is: > >> profile_simple_test_data: Control.Monad.Trans.Resource.stateCleanup: There >> is a bug in the implementation. The mutable state is being accessed after >> cleanup. Please contact the maintainers. > > A reduced code snippet that generates this error is (also attached): > >> import Control.Monad >> import System.Environment >> import Control.Monad.IO.Class (liftIO) >> import System.IO >> import Data.Conduit.Lazy >> import Data.List (sort) >> >> import Data.Conduit >> >> import Prelude hiding (map) >> >> main = getArgs >>= process >> >> process args = mapM_ sorted args >> >> sorted x = runResourceT (lazyConsume $ sourceFeed x) >>= (mapM_ print . id) >> >> sourceFeed :: ResourceIO m => FilePath -> Source m String >> sourceFeed file = sourceIO >> (openFile file ReadMode) >> hClose >> (\h -> liftIO $ do >> eof <- hIsEOF h >> if eof >> then return IOClosed >> else fmap IOOpen $ hGetLine h) > > when run over any text file. > > I may be doing something inconsistent with the correct use of sourceIO > or lazyConsume, however, I tried to follow the example at > http://www.yesodweb.com/home/snoyberg/blogs/conduit/conduit/source/source.ditamap?nav=nav-2 > as closely as possible. > > Is this a bug, or simply an incorrect use of Conduit? I haven't fully debugged this yet. There's certainly a bug in the implementation of ResourceT, but the sample program is also wrong. You can't pass the result from a call to lazyConsume outside the scope of its ResourceT; the correct way to write sorted would be: sorted x = runResourceT $ lazyConsume (sourceFeed x) >>= mapM_ (liftIO . print) My guess is that this is a fallout from the transition away from mutable variables: lazyConsume no longer has any way of knowing that its ResourceT has already been terminated. Perhaps a simple solution would be to expose a primitive that checks if the ResourceT block has already been finalized. Michael ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Conduit Error Output: Control.Monad.Trans.Resource.stateCleanup
Hi Michael, Café. I'm writing some code using the conduit library and am encountering the following error output (while the program appears to function correctly) when using Data.Conduit.Lazy. The error given is: > profile_simple_test_data: Control.Monad.Trans.Resource.stateCleanup: There is > a bug in the implementation. The mutable state is being accessed after > cleanup. Please contact the maintainers. A reduced code snippet that generates this error is (also attached): > import Control.Monad > import System.Environment > import Control.Monad.IO.Class (liftIO) > import System.IO > import Data.Conduit.Lazy > import Data.List (sort) > > import Data.Conduit > > import Prelude hiding (map) > > main = getArgs >>= process > > process args = mapM_ sorted args > > sorted x = runResourceT (lazyConsume $ sourceFeed x) >>= (mapM_ print . id) > > sourceFeed :: ResourceIO m => FilePath -> Source m String > sourceFeed file = sourceIO > (openFile file ReadMode) > hClose > (\h -> liftIO $ do > eof <- hIsEOF h > if eof > then return IOClosed > else fmap IOOpen $ hGetLine h) when run over any text file. I may be doing something inconsistent with the correct use of sourceIO or lazyConsume, however, I tried to follow the example at http://www.yesodweb.com/home/snoyberg/blogs/conduit/conduit/source/source.ditamap?nav=nav-2 as closely as possible. Is this a bug, or simply an incorrect use of Conduit? conduit_error.hs Description: Binary data ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
