Re: [jira] Resolved: (OLIO-38) Rails app aborts and core dumps when running on Thin on OpenSolaris

[Shanti Subramanyam - PAE]

Amanda or Will,
 Can you please add the OpenSolaris specific issues and 
workarounds/fixes to the rails setup.html doc ? Or maybe a better idea 
would be to put them in the FAQ and then add a ptr. to the setup.html.


Shanti

On 01/25/09 05:28 PM, Mandy Waite (JIRA) wrote:
     [ 
https://issues.apache.org/jira/browse/OLIO-38?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mandy Waite resolved OLIO-38.
-----------------------------

    Resolution: Fixed

Problem with Eventmachine, fixed and fix pushed out the the Eventmachine git 
repository. Fix will be in next Eventmachine release. Here's details of the fix 
and the Eventmachine patch:

Francis and I just spent some time investigating the issue and I think
we've finally solved it. This should also fix the long-standing
problem in the kqueue reactor.

Basically, the problem arises when send() fails with an EWOULDBLOCK or
EINPROGRESS. The data is not sent, but its also not put back into
OutboundPages, which causes OutboundDataSize to get out of sync and
triggers the assertion failure. The fix is to always put unsent data
back into the outbound buffer:

===================================================================
--- ed.cpp      (revision 674)
+++ ed.cpp      (working copy)
@@ -609,31 +609,38 @@
        assert (GetSocket() != INVALID_SOCKET);
        int bytes_written = send (GetSocket(), output_buffer, nbytes, 0);

-       if (bytes_written > 0) {
-               OutboundDataSize -= bytes_written;
-               if ((size_t)bytes_written < nbytes) {
-                       int len = nbytes - bytes_written;
-                       char *buffer = (char*) malloc (len + 1);
-                       if (!buffer)
-                               throw std::runtime_error ("bad alloc
throwing back data");
-                       memcpy (buffer, output_buffer + bytes_written, len);
-                       buffer [len] = 0;
-                       OutboundPages.push_front (OutboundPage (buffer, len));
-               }
+       bool err = false;
+       if (bytes_written < 0) {
+               err = true;
+               bytes_written = 0;
+       }

-               #ifdef HAVE_EPOLL
-               EpollEvent.events = (EPOLLIN | (SelectForWrite() ?
EPOLLOUT : 0));
-               assert (MyEventMachine);
-               MyEventMachine->Modify (this);
-               #endif
-               #ifdef HAVE_KQUEUE
-               if (SelectForWrite()) {
-                       MyEventMachine->ArmKqueueWriter (this);
-                       cerr << "POW\n";
-               }
-               #endif
+       assert (bytes_written >= 0);
+       OutboundDataSize -= bytes_written;
+       if ((size_t)bytes_written < nbytes) {
+               int len = nbytes - bytes_written;
+               char *buffer = (char*) malloc (len + 1);
+               if (!buffer)
+                       throw std::runtime_error ("bad alloc throwing
back data");
+               memcpy (buffer, output_buffer + bytes_written, len);
+               buffer [len] = 0;
+               OutboundPages.push_front (OutboundPage (buffer, len));
        }
-       else {
+
+
+       #ifdef HAVE_EPOLL
+       EpollEvent.events = (EPOLLIN | (SelectForWrite() ? EPOLLOUT : 0));
+       assert (MyEventMachine);
+       MyEventMachine->Modify (this);
+       #endif
+       #ifdef HAVE_KQUEUE
+       if (SelectForWrite()) {
+               MyEventMachine->ArmKqueueWriter (this);
+       }
+       #endif
+
+
+       if (err) {
                #ifdef OS_UNIX
                if ((errno != EINPROGRESS) && (errno != EWOULDBLOCK)
&& (errno != EINTR))
                #endif


Rails app aborts and core dumps when running on Thin on OpenSolaris
-------------------------------------------------------------------

                Key: OLIO-38
                URL: https://issues.apache.org/jira/browse/OLIO-38
            Project: Olio
         Issue Type: Bug
         Components: rails-app
        Environment: OpenSolaris 2008.11
           Reporter: Mandy Waite

Rails app crashes after a random period of time when running on Thin  on 
OpenSolaris. Error output by Thin is:
Assertion failed: nbytes > 0, file ed.cpp, line 622
Abort (core dumped)
Which indicates that the problem occurred within native code in the eventmachine gem. 
The assertion checks that there are bytes available to be written on a call to eventmachine's Write(). At the time of the crash OutboundPages.size() = 0 
OutboundPages is the deque used to queue pages of data for writing. 
Stack trace:
core 'core' of 13659:   /usr/ruby/1.8/bin/ruby /var/ruby/1.8/gem_home/bin/thin 
-e production s
 fecb17c5 _lwp_kill (1, 6, 8034760, fec5a02e) + 15
 fec5a03a raise    (6, 0, 80347b0, fec315ea) + 22
 fec3160a abort    (65737341, 6f697472, 6166206e, 64656c69, 626e203a, 73657479) 
+ f2
 fec3185a _assert  (feaa4c96, feaa4bf6, 26e, 0, 6d7a, a17edb8) + 82
 fea9c2dd _ZN20ConnectionDescriptor18_WriteOutboundDataEv (a17ed88, 8038b0c, 
8038b8c, fea9c548) + 391
 fea9c575 _ZN20ConnectionDescriptor5WriteEv (a17ed88, fed3f000, 8038b10, 
fec4ae14, 2d, 9a5a744) + 3d
 fea9ebe3 _ZN14EventMachine_t14_RunSelectOnceEv (9a5a720, bc00a, fecb0615, 
fea9ee91, 9a5a760, feab51b8) + 20b
 fea9eead _ZN14EventMachine_t8_RunOnceEv (9a5a720, 8039234, 0, feaa18dd) + 29
 feaa191e _ZN14EventMachine_t3RunEv (9a5a720, 480, 4166135, fea8eeba) + 4e
 fea8ef2d evma_run_machine (0, 808a254, 80391dc, 80681fc, 839c594, 808a254) + 81
 fea912ba _Z29t_run_machine_without_threadsm (839c594) + 16
 080681fc rb_call0 (839c454, 839c594, 5161, 5161, 0, 0) + 998
 08068dd2 rb_call  (839c454, 839c594, 5161, 0, 0, 2) + 196
 080627f9 rb_eval  (839c594, 83b218c) + 19ed
 08063241 rb_eval  (839c594, 83b2628) + 2435
 080686ca rb_call0 (839c454, 839c594, 1411, 1411, 0, 0) + e66
 08068dd2 rb_call  (839c454, 839c594, 1411, 0, 0, 0) + 196
 08062c99 rb_eval  (84281e8, 8425df8) + 1e8d
 0806ea95 block_pass (84281e8, 8425e0c) + 3a1
 080636c8 rb_eval  (84281e8, 8425fd8) + 28bc
 080686ca rb_call0 (8422414, 84281e8, 13c1, 13c1, 0, 0) + e66
 08068dd2 rb_call  (8422414, 84281e8, 13c1, 0, 0, 0) + 196
 08062c99 rb_eval  (842865c, 8280b88) + 1e8d
 080686ca rb_call0 (827ccf4, 842865c, 13c1, 13c1, 0, 0) + e66
 08068dd2 rb_call  (827ccf4, 842865c, 13c1, 0, 0, 0) + 196
 08062c99 rb_eval  (8430104, 842ee1c) + 1e8d
 080686ca rb_call0 (84286c0, 8430104, 13c1, 13c1, 0, 8041444) + e66
 08068dd2 rb_call  (84286c0, 8430104, 13c1, 0, 8041444, 1) + 196
 08068fbc rb_f_send (1, 8041440, 8430104) + f4
 0806820d rb_call0 (80c8bb0, 8430104, fd1, fd1, 1, 8041440) + 9a9
 08068dd2 rb_call  (80c8bb0, 8430104, fd1, 1, 8041440, 0) + 196
 08062c99 rb_eval  (8286cb8, 834ba90) + 1e8d
 0806344a rb_eval  (8286cb8, 834c2b0) + 263e
 080686ca rb_call0 (8286114, 8286cb8, 6351, 6351, 0, 0) + e66
 08068dd2 rb_call  (8286114, 8286cb8, 6351, 0, 0, 2) + 196
 080627f9 rb_eval  (8286cb8, 834c508) + 19ed
 080686ca rb_call0 (8286114, 8286cb8, 4ecf, 4ecf, 0, 0) + e66
 08068dd2 rb_call  (8286114, 8286cb8, 4ecf, 0, 0, 0) + 196
 08062c99 rb_eval  (80c78b4, 83d7978) + 1e8d
 0805dfeb eval_node (80c78b4, 83d7978) + 3f
 0806a819 rb_load  (83d7ba8, 0) + 391
 0806aaf4 rb_f_load (1, 80469c0, 80c78b4) + 48
 0806820d rb_call0 (80c8bb0, 80c78b4, 25c1, 25c1, 1, 80469c0) + 9a9
 08068dd2 rb_call  (80c8bb0, 80c78b4, 25c1, 1, 80469c0, 1) + 196
 08062a39 rb_eval  (80c78b4, 80b761c) + 1c2d
 0805dfeb eval_node (80c78b4, 80b761c) + 3f
 0805e6d6 ruby_exec_internal (808a254, 80c8c50, 0, 0, 0, a00000) + d6
 0805e74b ruby_exec (808a254, 8058cbe, 8047d2c, 8058cc6, feffb7e4, feffde70) + 
27
 0805e773 ruby_run (feffb7e4, feffde70, 0, 8047d2c, 8076d8d, feffb7e4) + 23
 08058cc6 main     (5, 8047d60, 8047d78) + 3a
 08058bfe _start   (5, 8047e1c, 8047e33, 8047e53, 8047e56, 8047e61) + 7a