I'm writing a network server with some specific requirements: - 5-50 clients connected (almost) permanently (maybe a bit more, but definitely not hundreds of them) - possibly thousands of requests per seconds - responses need to be returned within 5 seconds or the client will disconnect and complain
Currently I have a Master thread (which is basically the main thread) which is handling connections/disconnections, socket operations, sends parsed requests for processing to single Worker thread, sends responses to clients. Interaction with Worker is done via message passing. The problem with my approach is that I read as much data as possible from each ready client in order. As there are many requests this read phase might take a few seconds making the clients disconnect. Now I see 2 possible solutions: 1. Stay with the design I have, but change the workflow somewhat - instead of reading all the data from clients just read some requests and then send responses that are ready and repeat; the downside is that it's more complicated than current design, might be slower (more loop iterations with less work done in each iteration) and might require quite a lot of tweaking when it comes to how many requests/responses handle each time etc. 2. Create separate thread per each client connection. I think this could result in a nice, clean setup, but I see some problems: - I'm not sure how ~50 threads will do resource-wise (although they will probably be mostly waiting on Socket.select) - I can't initialize threads created via std.concurrency.spawn with a Socket object ("Aliases to mutable thread-local data not allowed.") - I already have problems with "interrupted system call" on Socket.select due to GC kicking in; I'm restarting the call manually, but TBH it sucks I have to do anything about that and would suck even more to do that with 50 or so threads If anyone has any idea how to handle the problems I mentioned or has any idea for more suitable design I would be happy to hear it. It's also possible I'm approaching the issue from completely wrong direction, so you can correct me on that as well. -- Marek Janukowicz