Hello Marc! Thank you for all your answers. I was locking the mutex while changing the watchers etc, but it seems it was a bit too late. ev_set_loop_release_cb() did the trick and now mutex gets locked right before any modifications are made.
Thank you very much! Kind regards, Dejan Markic ________________________________________ From: Marc Lehmann [[email protected]] Sent: Friday, May 22, 2015 4:02 PM To: Dejan Markic Cc: [email protected] Subject: Re: Some questions about libev On Fri, May 22, 2015 at 01:26:45PM +0000, Dejan Markic <[email protected]> wrote: > Once the thread starts, it create hiredis async handler and attaches libev > with its libev handler and sets up all the callbacks. > After all is initialized and connected, I send commands from main thread to > all servers via redisAsyncCommand. I don't know the details of that adapter library, so I can only comment generally: > > Questions: > > 1. Hiredis libev adapter uses two ev_io watchers, one for reading and one for > writing. It's simply calling start and stop on those watchers based on what > it tries to do. > This does not work for me, unless I register another ev_async watcher, > that fires everytime hiredis tries to start/stop the io_watcher. Is that > expected? I would say that once > libev goes into poll/select/etc wait, it will not move out of wait if you > stop the watcher, or start the stopped watcher, right? The question is, how do you call ev_io_start or stop when the thread is blocked? The only way to do that is from another thread, which requires locking, and from your description, you don't do any locking. In general, if you use multiple threads, you have to use some kind of locking (or another suitable synchronisation mechanism) on shared data, you cannot just access data structures from other threads without synchronising with them. If you do, then watchers not firing is the leats of your problems, because its easy to corrupt data structures when different threads have different ciews of them. There are many ways to achieve that, either you use an async watcher to wake up the thread that uses the ev loop (and in its callback, you can then start/stop other watchers), or you lock the event loop (see THREAD LOCKING EXAMPLE in the libev docs), or a combinatzion of both. > 2. I tried another approach I saw on the internet. Single ev_io > watcher, but everytime hiredis tries to stop/start read/write, I to > ev_io_stop(...) and then I set EV_READ and/or EV_WRITE flags via > ev_io_set and then ev_io_start again. Also in that case, I need ev_async > watcher to wakeup the libev. Would you that that single IO watcher > approach would be better? As long as you use ev_io_start/stop in the same thread aqs your other libev calls, thats fine, of course also without ev_async. As soon as you use another thread to manipulate the same libev loop, yoiu have to lock it or otherwise synchronise. > Both 1. and 2. I tried protecting changes in watchers with mutex, but no > success. How did you protect changes in the watchers? In general, you have to lock every call, including ev_run, so inside a watcher your loop will likely still be locked. > 3. My test program hangs after few minutes, maybe even few seconds (it's > working for atleast 1000 iterations, even 100000 sometimes) after I fire it > (sending SET/GET to one or many servers). It hangs in epoll_wait or select > (tried different EVBACKEND_*). It doesn't receive the ev_async event it > seems. I also have a feeling that there might be some memory corruptions, but > I cannot find any using efence or valgrind. Modern cpus are very asynchronous - hoping that memory somehow synchronises itself between different cores is likely going to result in unexpected behaviour like you describe. > I've enabled EV_VERIFY 3 and sometimes I get "libev: pending watcher not on > pending queue" when on EVBACKEND_SELECT. Same thing, if you don't lock shared data structures, corruption is almost guaranteed. > Any hints before I go and completely rewrite it? I don't know your exact design, so I don't know what you could/should redesign, if anything, but in general, if you use threads, you need to be extreemly careful and not forget locking. The documentation gives a complete example in the THREAD LOCKING EXAMPLE, which likely doesn't exactly match your use case, but it shows how to relinquish a mutex while libev polls for new events, which allows you to modify libevs watchers from other threads. These changes will only take effect on the next event loop iteration, so additionally, you need to have an async watcher to take up your thread from another thread. Depending on your design, it might be simpler to only have an ev_async watcher and do the required modifications in its callback. In the latter case, you might not need extra locking, as you are allowed to call ev_async_send from other threads without locking (you might need a mutex to protect any variables you want to pass to that async watcher of course). In general, with threads, do never assume that you can share data structures between threads without synchronisation or locking. -- The choice of a Deliantra, the free code+content MORPG -----==- _GNU_ http://www.deliantra.net ----==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [email protected] -=====/_/_//_/\_,_/ /_/\_\ _______________________________________________ libev mailing list [email protected] http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
