Hi Yuriv, in which environments can you reproduce it? I tried on OSX with mono 2.10 and 2.12 for half an hour each with no luck.
On Thu, Jul 19, 2012 at 3:55 AM, Yuriy Solodkyy <yu...@couldbedone.com>wrote: > Brett, > > "No completion callback" is what proves the problem, unless there is a > problem in the sample code itself. > > I tried to rebuild mono with thread/pool instead of epoll socket > implementation and still get the same problem. So, I concluded it is > not epoll specific problem. > > -yuriy > > > On Thu, Jul 19, 2012 at 12:08 AM, Brett Ernst <brett.e.er...@gmail.com> > wrote: > > I've had some strangeness with the thread pool in the past, but never > enough > > to get a solid, consistent repro that I could file a bug for. I don't > know > > if this is related or not, but I've actually seen a simple Timer fail to > > generate callbacks under very high load (and on old hardware). Again, not > > repro-able enough to file a bug for but enough to make me nervous. > > > > When I run your mono-socket-problem code, I start seeing the "No > completion > > callback" messages within 5 seconds and then regularly every 5-10 > seconds or > > so. I can't say for sure if the issues are related, but if they are, > this is > > the best repro I've seen. > > > > As you can imagine, I've grown a bit of a distrust for the threadpool and > > thus async socket operations. I put some effort into digging through the > > mono internals, but without a solid repro and lacking a good > understanding > > of the thread pool implementation, my ultimate solution was to give up > and > > stop using async sockets altogether. > > > > I took a different approach: I wrapped libev and POSIX sockets. Manos de > > Mono is an excellent example of this approach. So far, this has been rock > > solid and performs extremely well. Of course, the major downsides are: 1) > > it's platform-specific, and 2) totally single-threaded. I get around #2 > by > > simply running multiple load-balanced nodes, one for each core. I still > make > > light use of the thread pool for long-running operations that shouldn't > > block the message loop. > > > > I only throw this out there as a possible alternative if you don't have > any > > success resolving this issue. Our architecture fit very well into the > event > > loop paradigm, but that may not work for everyone. > > > > On Tue, Jul 17, 2012 at 7:47 AM, Greg Young <gregoryyou...@gmail.com> > wrote: > >> > >> Btw to avoid confusion and duplicated work if someone starts could we > just > >> sync a bit in this thread? > >> > >> On Tuesday, July 17, 2012, Greg Young wrote: > >>> > >>> Hey all. > >>> > >>> As this is a big issue for us and I feel a huge problem for mono in > >>> general at this point as it means sockets basically dont work which is > a > >>> strong point of unix environments, I would like to propose something I > have > >>> done in the past. I am willing to offer a bounty (personally) for a > working > >>> fix to this section of code of $500 usd (more if done quickly). > >>> > >>> Acceptance criteria is the included test working in a stable fashion in > >>> Linux / bsd but just Linux is acceptable as well, > >>> > >>> I honestly wish more people would do this kind of thing with OSS > >>> projects. > >>> > >>> Cheers, > >>> > >>> Greg > >>> > >>> On Saturday, July 7, 2012, Yuriy Solodkyy wrote: > >>>> > >>>> Hi Rodrigo, > >>>> > >>>> please find a small sample app at > >>>> https://github.com/ysw/mono-socket-problem > >>>> > >>>> This app can start in either server or client mode. These modes only > >>>> differ in whether it listens for connections on multiple ports or > >>>> connects to server on multiple ports. Upon connecting to or accepting > >>>> connection it immediately sends some data, and then sends next chunk > >>>> of data in response to any data received from the other side. There > >>>> are some random delays in code and we limit outgoing traffic on each > >>>> connection not to be significantly higher than inbound. > >>>> > >>>> There is also a separate thread which regularly checks status of every > >>>> connection and report any connections that are awaiting a callback, > >>>> but their status obtained with socket.poll is already READY. (A > >>>> several seconds delay is still allowed). > >>>> > >>>> See also the README file. > >>>> > >>>> > >>>> Also, it seems that constantly changing men/max threads in ThreadPool > >>>> increases probability of the problem. See code. > >>>> Please let me know if this sample app works for you. > >>>> > >>>> Hope it helps > >>>> > >>>> > >>>> Thank you, > >>>> Yuriy > >>>> > >>>> > >>>> We've been aware of such issues, could you file a bug and attach a > test > >>>> case with it please? > >>>> > >>>> This would really really help us fix it. > >>>> > >>>> On Wed, Jun 27, 2012 at 4:08 AM, Greg Young <gregoryyoung1 at > gmail.com> > >>>> wrote: > >>>> > >>>> > We are experiencing an issue with async behaviours in sockets (with > >>>> > SendAsync/callback not Begin/End). > >>>> > > >>>> > Our visible issue is that when in a send loop we will fail on our > >>>> > heartbeats. After debugging and counting calls into/out of > >>>> > SendAsync/callback we see that we are inside of a call to SendAsync > >>>> > (eg: it never returns, in our case for 10 seconds before we declare > >>>> > the socket dead). We can duplicate this fairly regularly on > >>>> > mac/bsd/linux though its nonconsistent (sometimes it may happen > >>>> > repeatedly other times it works fine). The code does not have such > >>>> > issues on MS CLR. We are also running on loopback so its unlikely > that > >>>> > an underlying network problem is causing the hang up. The code > itself > >>>> > is fairly straight forward (not that different than the MS example > of > >>>> > the API except that its fully async (separate send/receive loops > while > >>>> > the example is request/response)) > >>>> > > >>>> > I am pulling sources now to build latest but does anyone happen to > >>>> > know of known issues with this sort of thing? > >>>> > > >>>> > Cheers, > >>>> > > >>>> > Greg > >>>> > > >>>> > -- > >>>> > Le doute n'est pas une condition agréable, mais la certitude est > >>>> > absurde. > >>>> > _______________________________________________ > >>>> > Mono-devel-list mailing list > >>>> > Mono-devel-list at lists.ximian.com > >>>> > http://lists.ximian.com/mailman/listinfo/mono-devel-list > >>>> > > >>>> > >>>> -- > >>>> Yuriy Solodkyy > >>>> (y.solod...@gmail.com) > >>>> _______________________________________________ > >>>> Mono-devel-list mailing list > >>>> Mono-devel-list@lists.ximian.com > >>>> http://lists.ximian.com/mailman/listinfo/mono-devel-list > >>> > >>> > >>> > >>> -- > >>> Le doute n'est pas une condition agréable, mais la certitude est > absurde. > >> > >> > >> > >> -- > >> Le doute n'est pas une condition agréable, mais la certitude est > absurde. > >> > >> _______________________________________________ > >> Mono-devel-list mailing list > >> Mono-devel-list@lists.ximian.com > >> http://lists.ximian.com/mailman/listinfo/mono-devel-list > >> > > > > > > _______________________________________________ > > Mono-devel-list mailing list > > Mono-devel-list@lists.ximian.com > > http://lists.ximian.com/mailman/listinfo/mono-devel-list > > > > > > -- > Yuriy Solodkyy > (y.solod...@gmail.com) > _______________________________________________ > Mono-devel-list mailing list > Mono-devel-list@lists.ximian.com > http://lists.ximian.com/mailman/listinfo/mono-devel-list >
_______________________________________________ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list