RE: [Haskell-cafe] Garbage collection and finalizer thread priority
On 13 June 2005 14:32, Gracjan Polak wrote: > Simon Marlow wrote: >> >> I presume you're running GHC. There's no way to increase the >> priority of a thread - GHC's scheduler doesn't have a concept of >> priorities. >> > > Yes, I forgot to state it explicitly. > >> I would look into whether you can use mallocForeignPtr: this is much >> faster than using newForeignPtr with finalizerFree, because the >> memory is garbage collected directly and there's no need for a >> finalizer. > > I use mallocBytes + reallocBytes + addForeignPtrFinalizer > finalizerFree. I wouldn't think this is any different, thanks for > your suggestion. > > May I ask where that difference comes from? mallocBytes allocates memory using C's malloc(), which is expensive. Also, the finalizer to free the memory is also expensive. mallocForeignPtr on the other hand allocates memory directly in GHC's heap (much faster than malloc()), and the memory is garbage collected when it is no longer referenced, so there's no need for the finalizer. Still, ForeignPtr is quite a bit slower than Ptr. We have some ideas for improving it, which I'll hopefully get round to implementing sometime. Cheers, Simon ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
RE: [Haskell-cafe] Garbage collection and finalizer thread priority
On 13 June 2005 16:55, Sebastian Sylvan wrote: > On 6/13/05, Simon Marlow <[EMAIL PROTECTED]> wrote: > >> I presume you're running GHC. There's no way to increase the >> priority of a thread - GHC's scheduler doesn't have a concept of >> priorities. >> > > Just out of curiousity, what scheme does GHC use for scheduling > threads? Just round-robin, at the moment. Cheers, Simon ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] Garbage collection and finalizer thread priority
Hello Gracjan, Tuesday, June 14, 2005, 4:36:26 PM, you wrote: >> it's an error in GHC 6.4/win32 >> GP> :) GP> How do I take care of that? use `+RTS -M' to increase it. you can include this optiion in executable, so user don't need to specify it each time. see somewhere in GHC docs -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Garbage collection and finalizer thread priority
Bulat Ziganshin wrote: Hello Gracjan, Tuesday, June 14, 2005, 1:29:09 PM, you wrote: GP> Documentation says: GP> -Msize GP> [Default: unlimited] Set the maximum heap size to size bytes. The GP> heap normally grows and shrinks according to the memory requirements of GP> the program... GP> GHC 6.4 says: GP> Heap exhausted; GP> Current maximum heap size is 268435456 bytes (256 Mb); GP> use `+RTS -M' to increase it. it's an error in GHC 6.4/win32 :) How do I take care of that? -- Gracjan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] Garbage collection and finalizer thread priority
Hello Gracjan, Tuesday, June 14, 2005, 1:29:09 PM, you wrote: GP> Documentation says: GP> -Msize GP> [Default: unlimited] Set the maximum heap size to size bytes. The GP> heap normally grows and shrinks according to the memory requirements of GP> the program... GP> GHC 6.4 says: GP> Heap exhausted; GP> Current maximum heap size is 268435456 bytes (256 Mb); GP> use `+RTS -M' to increase it. it's an error in GHC 6.4/win32 -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Garbage collection and finalizer thread priority
Simon Marlow wrote: On 13 June 2005 11:30, Gracjan Polak wrote: My space problems continued... :) Follow up :) I have some ForeignPtr's, with finalizers (finalizerFree only). And I have lazyly produced list of those (yes, there is some unsafePerformIO magic behind, but I think I got this right). The problem is I get out-of-memory condition because elements get produced faster than those consumed are garbage collected. Example: list = ... mapM_ writeOut list writeOut :: Ptr Word8 writeOut dat = do hPutBuh handle dat 1024 -- Control.Concurrent.threadDelay 1000 This sould be: writeOut :: Ptr Word8 writeOut dat = do hPutBuh handle dat 1024 System.Mem.performGC Control.Concurrent.threadDelay 1000 My small experimets show, that gc is not triggered?!?! What are the conditions to trigger gc? As I read the docs, is some % of heap. Is mallocForeinPtrBytes counted into that %? Anyway I ended up triggering GC by hand and giving it time to run finalizers. Frankly speaking this is nonsolution:( Related question: Documentation says: -Msize [Default: unlimited] Set the maximum heap size to size bytes. The heap normally grows and shrinks according to the memory requirements of the program... GHC 6.4 says: Heap exhausted; Current maximum heap size is 268435456 bytes (256 Mb); use `+RTS -M' to increase it. What am I missing? PS: I might be mistaken in any of above statements. I'll be thankful for any light... :) -- Gracjan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Garbage collection and finalizer thread priority
Sebastian Sylvan wrote: On 6/13/05, Simon Marlow <[EMAIL PROTECTED]> wrote: I presume you're running GHC. There's no way to increase the priority of a thread - GHC's scheduler doesn't have a concept of priorities. Just out of curiousity, what scheme does GHC use for scheduling threads? As I read somewhere, round robin... -- Gracjan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Garbage collection and finalizer thread priority
On 6/13/05, Simon Marlow <[EMAIL PROTECTED]> wrote: > I presume you're running GHC. There's no way to increase the priority > of a thread - GHC's scheduler doesn't have a concept of priorities. > Just out of curiousity, what scheme does GHC use for scheduling threads? /S -- Sebastian Sylvan +46(0)736-818655 UIN: 44640862 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Garbage collection and finalizer thread priority
Simon Marlow wrote: I presume you're running GHC. There's no way to increase the priority of a thread - GHC's scheduler doesn't have a concept of priorities. Yes, I forgot to state it explicitly. I would look into whether you can use mallocForeignPtr: this is much faster than using newForeignPtr with finalizerFree, because the memory is garbage collected directly and there's no need for a finalizer. I use mallocBytes + reallocBytes + addForeignPtrFinalizer finalizerFree. I wouldn't think this is any different, thanks for your suggestion. May I ask where that difference comes from? -- Gracjan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
RE: [Haskell-cafe] Garbage collection and finalizer thread priority
On 13 June 2005 11:30, Gracjan Polak wrote: > My space problems continued... :) > > I have some ForeignPtr's, with finalizers (finalizerFree only). And I > have lazyly produced list of those (yes, there is some unsafePerformIO > magic behind, but I think I got this right). The problem is I get > out-of-memory condition because elements get produced faster than > those consumed are garbage collected. > > Example: > > list = ... > mapM_ writeOut list > > writeOut :: Ptr Word8 > writeOut dat = do > hPutBuh handle dat 1024 > -- Control.Concurrent.threadDelay 1000 > > Uncommenting this line allows gc thread to run finalizers, memory gets > freed, everything runs smoothly... > > As far as I know finalizers are run in separate thread. How do I > increase priority of this thread so it runs faster? I presume you're running GHC. There's no way to increase the priority of a thread - GHC's scheduler doesn't have a concept of priorities. I would look into whether you can use mallocForeignPtr: this is much faster than using newForeignPtr with finalizerFree, because the memory is garbage collected directly and there's no need for a finalizer. Cheers, Simon ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Garbage collection and finalizer thread priority
Hi, My space problems continued... :) I have some ForeignPtr's, with finalizers (finalizerFree only). And I have lazyly produced list of those (yes, there is some unsafePerformIO magic behind, but I think I got this right). The problem is I get out-of-memory condition because elements get produced faster than those consumed are garbage collected. Example: list = ... mapM_ writeOut list writeOut :: Ptr Word8 writeOut dat = do hPutBuh handle dat 1024 -- Control.Concurrent.threadDelay 1000 Uncommenting this line allows gc thread to run finalizers, memory gets freed, everything runs smoothly... As far as I know finalizers are run in separate thread. How do I increase priority of this thread so it runs faster? -- Gracjan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
