Over the past couple of months, I have been struggling with an issue where mono 2.10.8 would sometimes crash badly on a heavily loaded simulator (2000+ scripts). Symptoms are 100% cpu use, simulator locking up with all threads waiting on a lock which no other thread holds or crashing with native stack trace

From analysis, this seems to be due to the use of Thread.Abort() to terminate script threads whose events do not complete within a short (or sometimes no) timeout. This is used in llDie() and llResetOtherScript(), amongst other things.

The use of Thread.Abort() is considered harmful, leading to the kind of system instability described above [1] [2] [3], etc. This affects scripts running in both separate AppDomains and the same AppDomain, since script functions constantly reach into core OpenSimulator code.

One could extend the timeout, but there is a tradeoff with timely script stopping. Also, even a large timeout would not guarantee that threads would not be aborted.

Therefore, I will probably soon investigate a mechanism for stopping scripts without calling Thread.Abort(). This will probably involve the use of Thread.Interrupt() for Sleeps and the insertion of 'should I stop' flag checks in generated code where that code may be long running (e.g. control structures for, goto, etc. and user defined functions). This is the normal approach one would take to co-operatively stop threads and avoid the chance of leaving Mono/.NET in a bad state.

The existing Thread.Abort mechanisms will remain default until this approach is shown not to have significant bugs/problems. However, I think in the long term we have to stop using Thread.Abort() by default since it is not a safe thing to do.

Comments welcome.

[1] http://stackoverflow.com/questions/1051838/killing-a-thread-c
[2] http://msdn.microsoft.com/en-us/library/7a2f3ay4%28v=vs.80%29.aspx
[3] 
http://stackoverflow.com/questions/2251964/c-sharp-thread-termination-and-thread-abort

--
Justin Clark-Casey (justincc)
OSVW Consulting
http://justincc.org
http://twitter.com/justincc
_______________________________________________
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Reply via email to