Re: partial x86_64 binary snapshots?
Jens Petersen wrote: bunzip2: ghc-6.4.2.20060621-x86_64-unknown-linux.tar.bz2: file ends unexpectedly Both the 20060620 and 20060621 snapshots are incomplete but 20060619 is okay. Jens ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Re[2]: FFI: number of worker threads?
On Wed, 2006-06-21 at 17:55 +0100, Duncan Coutts wrote: > So I'd suggest the best approach is to keep the existing multiplexing > non-blocking IO system and start to take advantage of more scalable IO > APIs on the platforms we really care about (either select/poll > replacements or AIO). FYI: Felix provides a thread for socket I/O. Only one is required. Uses epoll/kqueue/io completion ports/select, depending on OS support. Similarly, there is one thread to handle timer events. We're working on a similar arrangement for asynchronous file I/O which many OS provide support for (Windows, Linux at least). There are only a limited number of devices you can connect to a computer. You can't need more than a handful of threads, unless the OS design is entirely lame .. in which case you'd not be trying to run high performance heavily loaded applications on that system in the first place. Our biggest headache is non-reentrant APIs such as Open GL which are only re-entrant on a process basis, this doesn't play with with pre-emptive threading and it's also not good for cooperative threading. The only real solution here is to run a server thread and a thread safe abstraction layer, which cooperate to do context switches when necessary. -- John Skaller Felix, successor to C++: http://felix.sf.net ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
partial x86_64 binary snapshots?
I wanted to download a nightly binary stable snapshot for Linux x86_64 to test if the problem with selinux on Fedora Core has been fixed... but AFAICT recent snapshots (I tried too) are broken: % bunzip2 -t ghc-6.4.2.20060621-x86_64-unknown-linux.tar.bz2 bunzip2: ghc-6.4.2.20060621-x86_64-unknown-linux.tar.bz2: file ends unexpectedly You can use the `bzip2recover' program to attempt to recover data from undamaged sections of corrupted files. Just wondering what is up with this? Jens ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re[4]: FFI: number of worker threads?
Hello Peng, Wednesday, June 21, 2006, 8:31:41 PM, you wrote: > My wish is to have a future GHC implementation that (a) uses blocking > I/O directly provided by the OS, and (b) provides more control over OS > threads and the internal worker thread pool. Using blocking I/O will > simplify the current design and allow the programmer to take advantage > of high-performance OS threads. If non-blocking I/O is really needed, > the programmer can use customized, Claessen-style threads wrapped in > modular libraries---some of my preliminary tests show that > Claessen-style threads can do a much better job to multiplex > asynchronous I/O. all I/O is done by the library procedures and so we can use other implementations (other libraries) without waiting for GHC changes (and GHC imho will be changed to include some sort of such library instead of adding new features to current already very complex I/O implementation) one of such libs is Einar's network-alt library that uses select/epoll/kqueue to overlap network i/o another library is my own Streams library that implements all the layers of I/O functionality and need only to implement read()/write() behavior in some way. currently is uses direct read()/write() calls but if someone will make alternative fdGetBuf/fdPutBuf implementations, it will work with it read: http://haskell.org/haskellwiki/Library/Streams Download: http://www.haskell.org/library/Streams.tar.gz Installation: run "make install" ps: new version of library is coming soon, but in this particular area nothing was changed -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: Re[2]: FFI: number of worker threads?
On Wed, 21 Jun 2006, Simon Peyton-Jones wrote: > New worker threads are spawned on as needed. You'll need as many of > them as you have simultaneously-blocked foreign calls. If you have 2000 > simultaneously-blocked foreign calls, you'll need 2000 OS threads to > support them, which probably won't work. Does the RTS use select() to multiplex network IO instead of spawning threads? Tony. -- f.a.n.finch <[EMAIL PROTECTED]> http://dotat.at/ ROCKALL MALIN: NORTHWESTERLY 7 TO SEVERE GALE 9 DECREASING 5 OR 6. RAIN OR SHOWERS. MODERATE OR GOOD, BECOMING POOR AT TIMES. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Re[2]: FFI: number of worker threads?
On 6/21/06, Duncan Coutts <[EMAIL PROTECTED]> wrote: On linux, epoll scales very well with minimal overhead. Using multiple OS threads to do blocking IO would not scale in the case of lots of idle socket connections, you'd need one OS thread per socket. On Linux, OS threads can also scale very well. I have done an experiment using pipes and NPTL where most connections are idle---the performance scales like a straight line when up to 32K file descriptors and 16K threads are used. The IO is actually no longer done inside the RTS, it's done by a Haskell worker thread. So it should be easier now to use platform-specific select() replacements. It's already different between unix/win32. So I'd suggest the best approach is to keep the existing multiplexing non-blocking IO system and start to take advantage of more scalable IO APIs on the platforms we really care about (either select/poll replacements or AIO). It is easy to take advantage of epoll---it shouldn't be that hard to bake it in. The question is about flexiblity: do we want it to be edge-triggered or level-triggered? Even with epoll built-in, the disk performance cannot keep up with NPTL unless AIO is also built-in. But for AIO, it is more complicated. It bypasses the OS caching; the Linux AIO even requires the use of certain kinds of file systems. My idea is that not everybody needs high-performance, asynchronous or nonblocking I/O. For those who really need it, it is worth (or, necessary) writing their own event loops, and event-driven programming in Haskell is not that difficult using CPS monads. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Re[2]: FFI: number of worker threads?
On Wed, 2006-06-21 at 12:31 -0400, Li, Peng wrote: > On 6/21/06, Simon Peyton-Jones <[EMAIL PROTECTED]> wrote: > > New worker threads are spawned on as needed. You'll need as many of > > them as you have simultaneously-blocked foreign calls. If you have 2000 > > simultaneously-blocked foreign calls, you'll need 2000 OS threads to > > support them, which probably won't work. > > 2000 OS threads definitely sound scary, but it is possible to work. > The Linux NPTL threads can scale well up to 10K threads and the stack > address spaces would be sufficient on 64-bit systems. > > I am thinking about some p2p applications where each peer is > maintaining a huge amount of TCP connections to other peers, but most > of these connections are idle. Unforturnately the default GHC RTS is > multiplexing I/O using "select", which is O(n) and it seems to have a > FDSET size limit of 1024. > > That makes me wonder if the current design of the GHC RTS is optimal > in the long run. As software and hardware evolves, we will have > efficient OS threads (like NPTL) and huge (64-bit) address spaces. > My guess is > > (1) It is always a good idea to multiplex GHC user-level threads on OS > threads, because it improve performance. Indeed. > (2) It may not be optimal to multiplex nonblocking I/O inside the GHC > RTS, because it is unrealistic to have an event-driven I/O interface > that is both efficient (like AIO/epoll) and portable (like > select/poll). What is worse, nonblocking I/O still blocks on disk > accesses. On the other hand, the POSIX threads are portable and it can > be efficiently implemented on many systems. At least on Linux, NPTL > easily beats "select"! On linux, epoll scales very well with minimal overhead. Using multiple OS threads to do blocking IO would not scale in the case of lots of idle socket connections, you'd need one OS thread per socket. The IO is actually no longer done inside the RTS, it's done by a Haskell worker thread. So it should be easier now to use platform-specific select() replacements. It's already different between unix/win32. So I'd suggest the best approach is to keep the existing multiplexing non-blocking IO system and start to take advantage of more scalable IO APIs on the platforms we really care about (either select/poll replacements or AIO). Duncan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Re[2]: FFI: number of worker threads?
On 6/21/06, Simon Peyton-Jones <[EMAIL PROTECTED]> wrote: New worker threads are spawned on as needed. You'll need as many of them as you have simultaneously-blocked foreign calls. If you have 2000 simultaneously-blocked foreign calls, you'll need 2000 OS threads to support them, which probably won't work. 2000 OS threads definitely sound scary, but it is possible to work. The Linux NPTL threads can scale well up to 10K threads and the stack address spaces would be sufficient on 64-bit systems. I am thinking about some p2p applications where each peer is maintaining a huge amount of TCP connections to other peers, but most of these connections are idle. Unforturnately the default GHC RTS is multiplexing I/O using "select", which is O(n) and it seems to have a FDSET size limit of 1024. That makes me wonder if the current design of the GHC RTS is optimal in the long run. As software and hardware evolves, we will have efficient OS threads (like NPTL) and huge (64-bit) address spaces. My guess is (1) It is always a good idea to multiplex GHC user-level threads on OS threads, because it improve performance. (2) It may not be optimal to multiplex nonblocking I/O inside the GHC RTS, because it is unrealistic to have an event-driven I/O interface that is both efficient (like AIO/epoll) and portable (like select/poll). What is worse, nonblocking I/O still blocks on disk accesses. On the other hand, the POSIX threads are portable and it can be efficiently implemented on many systems. At least on Linux, NPTL easily beats "select"! My wish is to have a future GHC implementation that (a) uses blocking I/O directly provided by the OS, and (b) provides more control over OS threads and the internal worker thread pool. Using blocking I/O will simplify the current design and allow the programmer to take advantage of high-performance OS threads. If non-blocking I/O is really needed, the programmer can use customized, Claessen-style threads wrapped in modular libraries---some of my preliminary tests show that Claessen-style threads can do a much better job to multiplex asynchronous I/O. If you think you have only a handful of simultaneously-blocked foreign calls, but you still get "runaway worker threads", please do make a reproducible test case and file a bug report. Yes, I will try to make a reproducible test case soon. Once you get answers, can I ask either or both of you to type in what you learned to the GHC user-documentation Wiki? That way things improve! The place to start is here http://haskell.org/haskellwiki/GHC under "Collaborative documentation". There's a already a page for "Concurrency" and for "FFI", so you can add to those. Thanks Certainly! ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: FFI: number of worker threads?
Simon, Thanks for the info. I don't compare thread IDs. At the moment I merely print out the thread ID in a trace message. Shortly I will be using the thread ID when a need arises to kill a thread. It sounds like the rollover is harmless for these situations. When you talk about comparing thread IDs, are you thinking that one might compare two thread IDs to see which one is more recently spawned? I can see where you might have a situation where you would compare thread IDs to determine whether two somehow related values "belong" in some sense to the same thread. I'm curious about why one might compare thread IDs in such a way that the rollover would cause the comparison to produce the "wrong" answer. Seth On Wed, 21 Jun 2006 12:48:42 +0100 Simon Marlow <[EMAIL PROTECTED]> wrote: > Seth Kurtzberg wrote: > > Another related question. I have some threaded applications running > > which are servers and run continuously. A thread is spawned for each > > new connection, and the thread exits when the client terminates. > > > > I've noticed that the thread ID increases. On one process I checked > > today I am up to thread number 3300. The number of running threads > > is not increasing; only six threads are running on this particular > process. > > The threads are cleaned up and exit. The thread _ID_ is continually > > increasing. > > > > Is this going to cause a problem when the thread ID exceeds some value? Do > > I have to force the server process to recycle periodically? > > > > These processes are designed to run continuously, and are running in a > > fairly demanding commercial environment for extended periods of time. They > > have proven to be very stable and reliable. I'm hopeful that it will not > > be necessary to recycle to force the thread ID to restart. > > The thread ID assigned to new threads will wrap around when it reaches > 2147483647. In 6.6 we made thread IDs 64 bits, so you get a bit longer > before they wrap around. Even if you manage to wrap the thread ID, > it'll only be a problem if you actually compare ThreadIDs. > > Cheers, > Simon > ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: FFI: number of worker threads?
Simon, Thanks for the response. The doc I was referring to is the library haddock doc for Control.Concurrent. Seth On Wed, 21 Jun 2006 12:41:52 +0100 Simon Marlow <[EMAIL PROTECTED]> wrote: > Seth Kurtzberg wrote: > > I have a related question. The docs state that in some environments O/S > > threads are used when the -threaded flag is used with ghc, and non-O/S > > threads are used otherwise (presumably these are non-preemptive). Does > > this apply as well to the worker threads that are the subject of this email? > > It sounds like the docs are a bit unclear. Which bit of doc in > particular are you referring to? > > forkIO always creates a lightweight thread. With -threaded, if a thread > makes a safe foreign call, then that call might execute concurrently > with other threads, because another OS thread (a worker thread) takes > over in the runtime. In the HEAD (which will be 6.6), we now allow > multiple OS threads in the runtime, so you also get to run multiple > Haskell threads simultaneously, which is particularly useful if you have > more than one CPU. > > Cheers, > Simon > ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: about threads (here: increasing thread IDs)
Jost, At the moment I am only using the thread ID in trace messages, so if I can safely ignore the overflow, then I will so. Seth On Wed, 21 Jun 2006 10:28:43 +0200 Jost Berthold <[EMAIL PROTECTED]> wrote: > Hi Seth, > > perhaps this will help you, some information from the implementation > side concerning your question about threadIDs: > > Related code lived in rts/Schedule.c, has recently moved to > rts/Threads.* in the head. > > The ThreadID assigned by the RTS is continuously increasing, and > declared to be StgInt32 (32 bits usually). > So you might get int overflows when running your server for a *fairly* > long time. OTOH, depending on what you want to do with the ThreadIDs > that forkIO returns, IMHO you could just *ignore* these overflows. > > From your short description of how the servers work, you know an upper > bound on the no. of running threads (6), but not on their time to live. > So IMHO the worst case scenario would be: > a new thread for a client gets ID n > ... 2^32 other server threads get spawned > another new thread for a client gets the same ID n again > ... > one of the threads supposed to terminate (killThread n) > > AFAICT, the killThread action will arbitrarily choose one of the two. > > HTH > Jost > > -- you wrote -- > Date: Tue, 20 Jun 2006 23:22:26 -0400 > From: Seth Kurtzberg <[EMAIL PROTECTED]> > Subject: Re: FFI: number of worker threads? > To: "Li, Peng" <[EMAIL PROTECTED]> > Cc: glasgow-haskell-users@haskell.org > > Another related question. I have some threaded applications running > which are servers and run continuously. A thread is spawned for each > new connection, and the thread exits when the client terminates. > > I've noticed that the thread ID increases. On one process I checked > today I am up to thread number 3300. The number of running threads is > not increasing; only six threads are running on this particular process. > The threads are cleaned up and exit. The thread _ID_ is continually > increasing. > > Is this going to cause a problem when the thread ID exceeds some value? > Do I have to force the server process to recycle periodically? > ... > > > ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
The GHC hackathon: 14,15 September 2006, Portland
Friends After some encouragement at last year's ICFP, Simon and I are going to run a GHC hackathon, in Portland, just before ICFP this September (14-15th). It'll be held at Galois's offices, in Beaverton; we're very grateful to Galois for hosting the meeting. The details are here: http://hackage.haskell.org/trac/ghc/wiki/Hackathon If you are interested in finding out a bit about how GHC works inside, then you should find the hackathon fun. It will certainly be informal and interactive. GHC is a 100% open-source project, and the more people that are involved the better it goes. If you think you might come, please take a look at the above page, and let us know by registering. See you soon! Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Illegal instruction in GHC 6.5 generated code.
(If this is the wrong list for this, please tell me. I tried
Glasgow-Haskell-Bugs, but message got hold in moderation)
Hello,
Using GHC 6.5 from darcs repo ("pulled" in 19/Jun), the following
snippet of code seems to cause illegal instruction error on the
generated code. The error disappears if I use the Dummy instance on
top of Refeable. The code is from my bindings to Judy library,
revision 10752 in subversion.
http://svn.openfoundry.org/pugs/third-party/HsJudy/
http://svn.openfoundry.org/pugs/third-party/judy/#
original C lib code
To test it you'll need to compile (./configure; make # but not
necessarily install) the original library in
third-party/judy/Judy-1.0.3. Back in HsJudy, change the Makefile to
static link option.
HsJudy$ make testmap
HsJudy$ ./testmap
Sorry, I tried to come up with an simpler instance of the problem but
didn't found one. The program depend on WordPtr which isn't available
in GHC 6.4.2, so I couldn't test it in stable version. It was tested
on my amd64, linux 2.6.16 and in audreyt's machine (OSX I think).
-8<
--class Dummy a
--instance Dummy a
--instance Dummy a => Refeable a where
instance Refeable a where
toRef a = do
a' <- newStablePtr a
return (ptrToWordPtr (castStablePtrToPtr a'))
fromRef v = do
a <- deRefStablePtr (castPtrToStablePtr (wordPtrToPtr v))
return a
-- Don't need to StablePtr "simple" things like Int
instance Refeable Int where
toRef i = return $ toEnum i
fromRef v = return $ fromEnum v
-->8-
Thanks in advance,
--
Caio Marcelo
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re[2]: FFI: number of worker threads?
Hello Simon, Wednesday, June 21, 2006, 4:10:46 PM, you wrote: | >> (page http://www.cse.unsw.edu.au/~chak/haskell/ghc/comm/ contains | >> commentaries about GHC internals) > | > | And ideally all that material should be on the wiki. > In fact we are looking for volunteers to do the HTML -> Wiki > translation. No new writing reqd! i thought about writing program that automates this task. it will be great for some my own docs. i asked once in haskell lists about such program but no one knows about this. may be someone can try to ask in more general forum? if such program can't be found, i will lazily volunteer to do it (not at this moment, but i have added it to my haskell to-do list) -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: FFI: number of worker threads?
| > (page http://www.cse.unsw.edu.au/~chak/haskell/ghc/comm/ contains | > commentaries about GHC internals) | | And ideally all that material should be on the wiki. In fact we are looking for volunteers to do the HTML -> Wiki translation. No new writing reqd! Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: FFI: number of worker threads?
Simon Marlow wrote: Seth Kurtzberg wrote: Another related question. I have some threaded applications running > which are servers and run continuously. A thread is spawned for each > new connection, and the thread exits when the client terminates. I've noticed that the thread ID increases. On one process I checked > today I am up to thread number 3300. The number of running threads > is not increasing; only six threads are running on this particular process. > The threads are cleaned up and exit. The thread _ID_ is continually > increasing. Is this going to cause a problem when the thread ID exceeds some value? Do I have to force the server process to recycle periodically? These processes are designed to run continuously, and are running in a fairly demanding commercial environment for extended periods of time. They have proven to be very stable and reliable. I'm hopeful that it will not be necessary to recycle to force the thread ID to restart. The thread ID assigned to new threads will wrap around when it reaches 2147483647. In 6.6 we made thread IDs 64 bits oops, I lie. It's still 32 bits in 6.6. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: FFI: number of worker threads?
Seth Kurtzberg wrote: Another related question. I have some threaded applications running > which are servers and run continuously. A thread is spawned for each > new connection, and the thread exits when the client terminates. I've noticed that the thread ID increases. On one process I checked > today I am up to thread number 3300. The number of running threads > is not increasing; only six threads are running on this particular process. > The threads are cleaned up and exit. The thread _ID_ is continually > increasing. Is this going to cause a problem when the thread ID exceeds some value? Do I have to force the server process to recycle periodically? These processes are designed to run continuously, and are running in a fairly demanding commercial environment for extended periods of time. They have proven to be very stable and reliable. I'm hopeful that it will not be necessary to recycle to force the thread ID to restart. The thread ID assigned to new threads will wrap around when it reaches 2147483647. In 6.6 we made thread IDs 64 bits, so you get a bit longer before they wrap around. Even if you manage to wrap the thread ID, it'll only be a problem if you actually compare ThreadIDs. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: FFI: number of worker threads?
Bulat Ziganshin wrote: Wednesday, June 21, 2006, 7:18:48 AM, you wrote: Seth and Li, look at http://www.cse.unsw.edu.au/~chak/haskell/ghc/comm/rts-libs/multi-thread.html That's a good page, but it's about 2 generations out of date unfortunately. Some of it is relevant, but many of the details have changed. We do plan to write up the current runtime architecture, probabliy for a paper at some point. it may answer some of your questions (page http://www.cse.unsw.edu.au/~chak/haskell/ghc/comm/ contains commentaries about GHC internals) And ideally all that material should be on the wiki. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: FFI: number of worker threads?
Seth Kurtzberg wrote: I have a related question. The docs state that in some environments O/S threads are used when the -threaded flag is used with ghc, and non-O/S threads are used otherwise (presumably these are non-preemptive). Does this apply as well to the worker threads that are the subject of this email? It sounds like the docs are a bit unclear. Which bit of doc in particular are you referring to? forkIO always creates a lightweight thread. With -threaded, if a thread makes a safe foreign call, then that call might execute concurrently with other threads, because another OS thread (a worker thread) takes over in the runtime. In the HEAD (which will be 6.6), we now allow multiple OS threads in the runtime, so you also get to run multiple Haskell threads simultaneously, which is particularly useful if you have more than one CPU. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: FFI: number of worker threads?
Li, Peng wrote: The paper "Extending the Haskell FFI with Concurrency" mentioned the following in Section 6.3: "GHC's run-time system employs one OS thread for every bound thread; additionally, there is a variable number of so-called "worker" OS threads that are used to execute the unbounded (lightweight) threads." How does the runtime system determine the number of worker threads? Is the number hardcoded in the RTS or dynamically adjustable? Can a programmer specify it as an RTS option or change it using an API? > I would like to use a large number (say, 2000) of unbounded threads, each calling a blocking, safe foreign function via FFI import. What is supposed to happen if all the worker threads are used up? I tried this in the recent GHC 6.5 and got some kind of "runaway worker threads?" RTS failure message when more than 32 threads are used. Is it a current limitation of the RTS, or should I file a bug report for it? As mentioned by Simon PJ, the number of worker threads grows as needed (but doesn't decrease - that would be a useful improvement). The message you're seeing is due to an arbitrary limit I put on the number of workers in order to catch bugs in the runtime before they kill the machine. Perhaps I should remove the limit, or at least make it very much larger. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: about threads (here: increasing thread IDs)
Hi Seth, perhaps this will help you, some information from the implementation side concerning your question about threadIDs: Related code lived in rts/Schedule.c, has recently moved to rts/Threads.* in the head. The ThreadID assigned by the RTS is continuously increasing, and declared to be StgInt32 (32 bits usually). So you might get int overflows when running your server for a *fairly* long time. OTOH, depending on what you want to do with the ThreadIDs that forkIO returns, IMHO you could just *ignore* these overflows. From your short description of how the servers work, you know an upper bound on the no. of running threads (6), but not on their time to live. So IMHO the worst case scenario would be: a new thread for a client gets ID n ... 2^32 other server threads get spawned another new thread for a client gets the same ID n again ... one of the threads supposed to terminate (killThread n) AFAICT, the killThread action will arbitrarily choose one of the two. HTH Jost -- you wrote -- Date: Tue, 20 Jun 2006 23:22:26 -0400 From: Seth Kurtzberg <[EMAIL PROTECTED]> Subject: Re: FFI: number of worker threads? To: "Li, Peng" <[EMAIL PROTECTED]> Cc: glasgow-haskell-users@haskell.org Another related question. I have some threaded applications running which are servers and run continuously. A thread is spawned for each new connection, and the thread exits when the client terminates. I've noticed that the thread ID increases. On one process I checked today I am up to thread number 3300. The number of running threads is not increasing; only six threads are running on this particular process. The threads are cleaned up and exit. The thread _ID_ is continually increasing. Is this going to cause a problem when the thread ID exceeds some value? Do I have to force the server process to recycle periodically? ... ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: Re[2]: FFI: number of worker threads?
New worker threads are spawned on as needed. You'll need as many of them as you have simultaneously-blocked foreign calls. If you have 2000 simultaneously-blocked foreign calls, you'll need 2000 OS threads to support them, which probably won't work. If you think you have only a handful of simultaneously-blocked foreign calls, but you still get "runaway worker threads", please do make a reproducible test case and file a bug report. Simon M will probably reply to Seth's qns about thread IDs (I assume you mean Haskell thread ID?) in due course. Once you get answers, can I ask either or both of you to type in what you learned to the GHC user-documentation Wiki? That way things improve! The place to start is here http://haskell.org/haskellwiki/GHC under "Collaborative documentation". There's a already a page for "Concurrency" and for "FFI", so you can add to those. Thanks Simon | -Original Message- | From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] | On Behalf Of Bulat Ziganshin | Sent: 21 June 2006 07:00 | To: Seth Kurtzberg | Cc: glasgow-haskell-users@haskell.org | Subject: Re[2]: FFI: number of worker threads? | | Hello Seth, | | Wednesday, June 21, 2006, 7:18:48 AM, you wrote: | | | Seth and Li, look at http://www.cse.unsw.edu.au/~chak/haskell/ghc/comm/rts-libs/multi-thread. html | | it may answer some of your questions | | (page http://www.cse.unsw.edu.au/~chak/haskell/ghc/comm/ contains | commentaries about GHC internals) | | | > I have a related question. The docs state that in some | | >> The paper "Extending the Haskell FFI with Concurrency" mentioned the | | | -- | Best regards, | Bulatmailto:[EMAIL PROTECTED] | | ___ | Glasgow-haskell-users mailing list | Glasgow-haskell-users@haskell.org | http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: Would you like a job working on GHC?
Folks If you are interested in working on GHC as a job, now is the time to apply. We're going to start reviewing applications on Monday 26th. [I obviously worded his advert badly -- quite a few replies ended up in my spam filter :-) but don't worry I've dug them all out.] Simon | -Original Message- | From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] | On Behalf Of Simon Peyton-Jones | Sent: 09 June 2006 08:45 | To: haskell@haskell.org; GHC users; [EMAIL PROTECTED]; [EMAIL PROTECTED] | Subject: Would you like a job working on GHC? | | | Would you like to be paid to work on GHC? | | The Glasgow Haskell Compiler (GHC) is now being used by so many people, | on so many platforms, that at GHC HQ we've been struggling to keep up. | We're delighted to say that we are now looking to hire a GHC support | engineer to work with us. | | Here is how we envisage working together: | * You would work 3-5 days a week on GHC (details flexible). | * You don't need to live in Cambridge, but we would like you to visit | every few months, so that we build up a face-to-face relationship. | * You'll be a self-employed consultant, not a Microsoft employee. |You bill us; we pay you. | * The position is a one year fixed term contract in the first instance. | * GHC is, and will remain, an open-source project with a BSD-style | license. | | We are looking for help with the role of supporting GHC in the field. | Specifically, here's what the job involves: | * Investigating, prioritising and fixing bugs, improving the test suite | * Managing the STABLE branch of the source tree | * Managing the GHC release cycle | * Keeping the GHC web site up to date | * Answering questions from GHC users | * Helping to support the open-source community that works on GHC itself | | We're particularly looking for someone who is experienced in a variety | of operating system platforms and libraries. You should be able to deal | with questions like "I try to build GHC on Solaris 2.3.4 and get | `undefined symbol _readline'"; or "How do I link to DLLs on Windows?"; | or "does the ByteString library in GHC 6.2.1 do XXX?". The role will | not primarily involve working on GHC's core; it's the parts round the | edges that you would mainly focus on. You must be able to read and | write Haskell, but we don't expect that you'll need to write a great | deal of Haskell code. C and 'make' experience are important too. | | Above all, we are looking for someone who is enthusiastic about Haskell, | and fired up about the prospect of becoming a GHC expert. | | If you are interested, please send your CV and a statement of why you | would be a good fit for the job, to our Human Resources Department at | [EMAIL PROTECTED] We're happy to accept informal enquiries of | course: please contact Simon Marlow ([EMAIL PROTECTED]) or Simon | Peyton Jones ([EMAIL PROTECTED]) for further information. | | The rate of pay is dependent on qualifications and experience. It'll be | fun! | | Simon Peyton Jones and Simon Marlow ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
