Re: [Haskell-cafe] Question about concurrency, threads and GC
Hi! Thanks to all who responded! I got a lot of information to read and think about. For now I decided to use stm-channelize as the simplest approach which seem to be enough. On Mon, Mar 5, 2012 at 9:50 PM, Alexander V Vershilov wrote: > Hello. > > I've also written simple chat server based on conduits and stm channels > > https://github.com/qnikst/chat-server/blob/master/src/Main.hs > > it has quite similar aproach and maybe this solution can be used together > to have better results. > > -- > Alexander Vershilov > > Sat, Mar 03, 2012 at 02:05:17AM -0500, Joey Adams wrote >> On Fri, Mar 2, 2012 at 7:34 PM, Joey Adams >> wrote: >> > I'll try to put together a simple chat server example, like the one I >> > wrote for stm-channelize. >> >> Here it is: >> >> https://github.com/joeyadams/haskell-chat-server-example >> >> See, in particular, the serveLoop function. When a message is >> received from the client, it is written to the send channel of every >> other client. When a message is written on the client's own send >> channel, it is transmitted to the client. The primary thread for the >> client waits until one of the worker threads signals completion, then >> kills both of the worker threads. >> >> I hope this example gives you some ideas. >> >> -Joey > > ___ > Haskell-Cafe mailing list > Haskell-Cafe@haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe > ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Question about concurrency, threads and GC
Hello. I've also written simple chat server based on conduits and stm channels https://github.com/qnikst/chat-server/blob/master/src/Main.hs it has quite similar aproach and maybe this solution can be used together to have better results. -- Alexander Vershilov Sat, Mar 03, 2012 at 02:05:17AM -0500, Joey Adams wrote > On Fri, Mar 2, 2012 at 7:34 PM, Joey Adams wrote: > > I'll try to put together a simple chat server example, like the one I > > wrote for stm-channelize. > > Here it is: > > https://github.com/joeyadams/haskell-chat-server-example > > See, in particular, the serveLoop function. When a message is > received from the client, it is written to the send channel of every > other client. When a message is written on the client's own send > channel, it is transmitted to the client. The primary thread for the > client waits until one of the worker threads signals completion, then > kills both of the worker threads. > > I hope this example gives you some ideas. > > -Joey signature.asc Description: Digital signature ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Question about concurrency, threads and GC
On Sat, Mar 3, 2012 at 12:50 PM, Alp Mestanogullari wrote: > That's exactly what I would have needed, several times in the past 2 > years. I've been wondering about a good way to abstract this to have a > library where you'd just plug your "business logic" and thus not have to > care anymore about the implementation details, once and for all. > It's hard to know which abstraction might be sufficiently magical to allow you to skip caring about the implementation details; personally I think stm is pretty magical already :) G -- Gregory Collins ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Question about concurrency, threads and GC
Hi, On Sat, Mar 3, 2012 at 10:38 AM, Gregory Collins wrote: > Hi, > > The tutorial I gave for CUFP 2011 was a multi-user web chat program using > the Snap Framework. STM channels make this kind of problem super-easy to > deal with. Don't be afraid of forking lots of Haskell threads for programs > like this, because they're "green" threads, not OS threads (i.e. Haskell > threads are M:N multiplexed onto OS threads) and as such they have very > little overhead. > > Maybe you'll find the code interesting: > https://github.com/snapframework/cufp2011. The "business logic" of using > STM channels is here: > https://github.com/snapframework/cufp2011/blob/master/src/Snap/Chat/ChatRoom.hs That's exactly what I would have needed, several times in the past 2 years. I've been wondering about a good way to abstract this to have a library where you'd just plug your "business logic" and thus not have to care anymore about the implementation details, once and for all. I don't have the feeling stm-channelize is the best we can achieve. This really can be made simpler from the user's point of view. -- Alp Mestanogullari ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Question about concurrency, threads and GC
On Fri, Mar 2, 2012 at 5:19 PM, Paul Graphov wrote: > Hello Cafe! > > I am trying to implement networked application in Haskell. It should > accept many > client connections and support bidirectional conversation, that is not > just loop with > Request -> Response function but also sending notifications to clients etc. > Hi, The tutorial I gave for CUFP 2011 was a multi-user web chat program using the Snap Framework. STM channels make this kind of problem super-easy to deal with. Don't be afraid of forking lots of Haskell threads for programs like this, because they're "green" threads, not OS threads (i.e. Haskell threads are M:N multiplexed onto OS threads) and as such they have very little overhead. Maybe you'll find the code interesting: https://github.com/snapframework/cufp2011. The "business logic" of using STM channels is here: https://github.com/snapframework/cufp2011/blob/master/src/Snap/Chat/ChatRoom.hs G -- Gregory Collins ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Question about concurrency, threads and GC
On Fri, Mar 2, 2012 at 7:34 PM, Joey Adams wrote: > I'll try to put together a simple chat server example, like the one I > wrote for stm-channelize. Here it is: https://github.com/joeyadams/haskell-chat-server-example See, in particular, the serveLoop function. When a message is received from the client, it is written to the send channel of every other client. When a message is written on the client's own send channel, it is transmitted to the client. The primary thread for the client waits until one of the worker threads signals completion, then kills both of the worker threads. I hope this example gives you some ideas. -Joey ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Question about concurrency, threads and GC
Fri, Mar 02, 2012 at 07:34:41PM -0500, Joey Adams wrote > On Fri, Mar 2, 2012 at 3:04 PM, Alexander V Vershilov > wrote: > > Hello, Paul. > > > > It seems you should not use 3 threads, but run in a data-driven behaviour > > with > > one thread per client. > > I don't think this will work for Paul's situation. He needs to be > able to send notifications to clients. This means sending to the > client even when the client has nothing to say at the moment. > > I've been grappling with the same problem. See: > > http://www.haskell.org/pipermail/haskell-cafe/2012-January/098495.html > > The discussion was recently summed up in Parallel Haskell Digest 8 > (scroll down to "How to make asynchronous I/O composable and safe?"): > > http://www.well-typed.com/blog/64 > > To try to address this problem, I wrote a module called stm-channelize: > > http://hackage.haskell.org/package/stm-channelize > > However, I did not end up using it. I would point you to the source > of my channelize function, but I think it's more complicated than it > needs to be. > > I, for one, would recommend a three-thread approach. That is, for > each client, you basically have: > > * A thread for receiving > > * A thread for sending > > * A main thread that waits for the other two threads, and kills them > when one of them finishes or dies. > > I'll try to put together a simple chat server example, like the one I > wrote for stm-channelize. > > -Joey Yes, there is no way to use only one thread for such a case, but conduit aproach seems quite easy and take care about resources. Recently I wrote a question about such a situation [1] and it simplification [1] http://www.haskell.org/pipermail/haskell-cafe/2012-February/099770.html I've used smth like stm-channelize approach to make "virtual socket" when there was many clients on one connection. I can check stm-channelize more precietly and write if my aproach has usefull idea for that case. -- Alexander Vershilov. signature.asc Description: Digital signature ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Question about concurrency, threads and GC
On Fri, Mar 2, 2012 at 3:04 PM, Alexander V Vershilov wrote: > Hello, Paul. > > It seems you should not use 3 threads, but run in a data-driven behaviour with > one thread per client. I don't think this will work for Paul's situation. He needs to be able to send notifications to clients. This means sending to the client even when the client has nothing to say at the moment. I've been grappling with the same problem. See: http://www.haskell.org/pipermail/haskell-cafe/2012-January/098495.html The discussion was recently summed up in Parallel Haskell Digest 8 (scroll down to "How to make asynchronous I/O composable and safe?"): http://www.well-typed.com/blog/64 To try to address this problem, I wrote a module called stm-channelize: http://hackage.haskell.org/package/stm-channelize However, I did not end up using it. I would point you to the source of my channelize function, but I think it's more complicated than it needs to be. I, for one, would recommend a three-thread approach. That is, for each client, you basically have: * A thread for receiving * A thread for sending * A main thread that waits for the other two threads, and kills them when one of them finishes or dies. I'll try to put together a simple chat server example, like the one I wrote for stm-channelize. -Joey ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Question about concurrency, threads and GC
Hello, Paul. It seems you should not use 3 threads, but run in a data-driven behaviour with one thread per client. You can take a look at network-conduit [1] example, there is a very good aproach to tcp-network application. There are some iteratee based examples but they are not so extensible. [1] http://hackage.haskell.org/package/network-conduit -- Alexander Vershilov Fri, Mar 02, 2012 at 08:19:56PM +0400, Paul Graphov wrote > Hello Cafe! > > I am trying to implement networked application in Haskell. It should accept > many > client connections and support bidirectional conversation, that is not > just loop with > Request -> Response function but also sending notifications to clients etc. > > NB: I came from C++ background and used to explicit asynchonous operations > and pools of OS threads. (Boost.Asio + Boost.Thread) > > There seem to be lots of goodies around (green threads, STM). The > problem is that > I cannot figure out how to design such applications. First of all now > I have to start > some threads for each client. Client state and behaviour may change depending > on > data that comes from it or other clients. So it seems that I have to > start at least three > threads: one that reads data from socket, one that sends queued messages to > the socket and one that represents clients main behaviour loop. The > last one will > supposedly read from some STM sources, one of them being TChan/TMVar filled > by thread that reads socket. Here I get stuck: how to manage that threads? > > I tried to search STM examples at hackage and the most interesting > packages were: > > 1) combinatorrent - quite complex application, but it seem to be > highly Erlang-inspired, > so I am not sure that it is the simplest way of doing thing. > > 2) network-connection package - it is a library that wraps socket and > uses STM internally > but exposes almost the same interface with blocking operations. Thread > management is > quite explicit there. But this package is marked obsolete. > > I wondered if one can rely upon garbage collector to reap dead threads > when using STM? > Or is it too error prone (accidental references to STM object that > prevent it from being > collected) and unpredictable? > > I have read this post: > http://blog.ezyang.com/2011/07/blockedindefinitelyonmvar/ > > It shows that relying upon handling BlockedIndefinitelyOn* exceptions > is a bad idea but > is it OK just to rely upon that blocked forever thread will die and > not to kill it explicitly? > > And also if it is really reliable, what are exact rules when some > object goes out of scope > and becomes collectable? For example in blog post mentioned above in > "main1" example > "lock" variable is accessible until the end of do-block, i.e. if I add > "takeMVar lock" line to > its end, it will compile. But it is not used actually. Is it > guaranteed that explicit performGC > will reap it? > > Possibly the best answer would be just pointing me to some recent > examples of concurrent > network applications. > > Thanks! > > ___ > Haskell-Cafe mailing list > Haskell-Cafe@haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe signature.asc Description: Digital signature ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Question about concurrency, threads and GC
Hello Cafe! I am trying to implement networked application in Haskell. It should accept many client connections and support bidirectional conversation, that is not just loop with Request -> Response function but also sending notifications to clients etc. NB: I came from C++ background and used to explicit asynchonous operations and pools of OS threads. (Boost.Asio + Boost.Thread) There seem to be lots of goodies around (green threads, STM). The problem is that I cannot figure out how to design such applications. First of all now I have to start some threads for each client. Client state and behaviour may change depending on data that comes from it or other clients. So it seems that I have to start at least three threads: one that reads data from socket, one that sends queued messages to the socket and one that represents clients main behaviour loop. The last one will supposedly read from some STM sources, one of them being TChan/TMVar filled by thread that reads socket. Here I get stuck: how to manage that threads? I tried to search STM examples at hackage and the most interesting packages were: 1) combinatorrent - quite complex application, but it seem to be highly Erlang-inspired, so I am not sure that it is the simplest way of doing thing. 2) network-connection package - it is a library that wraps socket and uses STM internally but exposes almost the same interface with blocking operations. Thread management is quite explicit there. But this package is marked obsolete. I wondered if one can rely upon garbage collector to reap dead threads when using STM? Or is it too error prone (accidental references to STM object that prevent it from being collected) and unpredictable? I have read this post: http://blog.ezyang.com/2011/07/blockedindefinitelyonmvar/ It shows that relying upon handling BlockedIndefinitelyOn* exceptions is a bad idea but is it OK just to rely upon that blocked forever thread will die and not to kill it explicitly? And also if it is really reliable, what are exact rules when some object goes out of scope and becomes collectable? For example in blog post mentioned above in "main1" example "lock" variable is accessible until the end of do-block, i.e. if I add "takeMVar lock" line to its end, it will compile. But it is not used actually. Is it guaranteed that explicit performGC will reap it? Possibly the best answer would be just pointing me to some recent examples of concurrent network applications. Thanks! ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
