Re: [Zope-dev] Zope 2.12.17 does not die when I hit ctrl+C
On Fri, May 6, 2011 at 11:40 AM, Hanno Schlichting wrote: > Are you using zope.sendmail's mail:queuedDelivery in the project? If > yes, upgrade to zope.sendmail 3.5.3, which will make it into Zope > 2.12.18. Earlier versions registered an atexit handler that wasn't > compatible with Python 2.5+. The bug was only fixed in later versions > of zope.sendmail but not backported to the 3.5 stable branch. > I accidentally sent my previous reply to Hanno directly. But I have since bumped the zope.sendmail version in my versions.cfg file and ran buildout again. I can confirm that the problem was zope.sendmail. Thanks for the very prompt reply. You saved me a lot of debugging time. regards, Izak Burger ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope 2.12.17 does not die when I hit ctrl+C
Hi all, After a recent project based on the latest plone, which pulled in zope 2.12.17 as an egg, I find that when I run zope in foreground mode and hit ctrl+C, it does not die. It prints the expected "Shutting down" message, and the parent process dies, but the one that holds on to port 8080 stays alive and will not die unless I send it SIGKILL. The little bit of debugging I've done so far suggests that after the signal handler for SIGINT runs, SystemExit is raised but it is somehow ignored. After that it just drops back into the polling loop in Lifetime.lifetime_loop. Has anyone else noticed this odd behaviour? SIGKILL seems like a very bad idea as it does not give the process a chance to clean up, so I'm not particularly happy about what I need to do at the moment, even if its just development. regards, Izak Burger ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope closes connection if the client closes the write-end of connection
Leonardo Rochael Almeida wrote: > Going out on a limb here, but I think Varnish might be trying to save > on file-descriptors. Interestingly, The Squid FAQ almost seems to imply that closing the write-side can cause more file-descriptors to be used. Squid can apparently not tell the difference between half-closed and full-closed connections, and it makes matter worse. Though I do not know this for sure, some reading on the subject suggests that nginx also doesn't like half-closed connections. ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope closes connection if the client closes the write-end of connection
Tres Seaver wrote: > You might also look at "fixing" varnish: I don't know of any valid > reason for it to be using the "half-open" connection model to test that > an HTTP-based backend is "up" -- certainly no browser in the world does > that; instead, modern browsers nearly always try to keep the connection > open for subsequent requests. I have already "fixed" varnish, I commented out the shutdown() call. It now works as expected. I just had a discussion about this with a colleague and it appears unclear exactly where to blame this. When a client half-closes its connection while the server was calling recv(), it makes absolute sense that recv() SHOULD return an empty buffer. There is nothing to return, and there won't ever be, the connection has been closed. Python is therefore not to blame, even if it doesn't specifically check all the possible revents returned by poll(). When asyncore receives an empty result from recv(), it does correctly assume that the connection was shut down. It doesn't seem wrong for asyncore to let the upper layers know about this. It would seem that asyncore is not to blame either. When asyncore calls the close() method inside zope, it ends up shuting down the whole thing. Though I could argue that zope shouldn't be doing this, the idea of a half-open connection doesn't work all that well in python/asyncore anyway, so given the framework within which zope operates, zope isn't to blame either. Given that no browser (that I know of) does this half-closing thing, I would argue that varnish should at the very least offer a shutdown option for probing, so as to appear more like a browser. In addition, HTTP 1.1 usually leaves the connection open unless you ask for it to be closed, so it almost seems more common not to shutdown the one end. It seems then that the only way to "fix" this is to either put zope behind apache or something else that can handle half-closing, or to "fix" varnish. regards, Izak ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope closes connection if the client closes the write-end of connection
Hi again, Izak Burger wrote: > I eventually distilled the check varnish uses into a small C program, > and an interesting problem shows up. When you call shutdown(fd, SHUT_WR) > on your socket connection, in effect telling zope that you're done > talking to it, it looks like zope responds in kind by not talking to you > either. I deduce this from the fact that poll() returns POLLHUP in > revents and read() returns zero (EOF). POLLHUP, if I understand this > correctly, means the other side (zope) has hung up. It seems this might not be a bug in zope. I'd like to describe what I've found this morning and where this is most possibly going wrong. Please tell me if I'm mistaken about something. This goes all the way to what happens when you call poll() on a file descriptor. If you ask to be notified for POLLIN events only, it does not necessarily guarantee that POLLIN (or a timeout) are the only events that can cause poll() to return. The man page does not properly document this, but the header file does: /* Event types always implicitly polled for. These bits need not be set in `events', but they will appear in `revents' to indicate the status of the file descriptor. */ #define POLLERR 0x008 /* Error condition. */ #define POLLHUP 0x010 /* Hung up. */ #define POLLNVAL0x020 /* Invalid polling request. */ In other words, a POLLHUP event can cause your poll() call to return even though the POLLIN event you were looking for didn't occur. If we inspect Modules/socketmodule.c in the python (2.4.4) source code we see that it is assumed that if poll() returns a value greater than zero, it means data is available for reading. If the client calls shutdown(SHUT_WR), this will cause a POLLHUP, which fools python into thinking there is data to be read. When it attempts to read that data, it ends up with nothing, that is, recv() returns 0. Moving on to asyncore.py, we see this: data = self.socket.recv(buffer_size) if not data: # a closed connection is indicated by signaling # a read condition, and having recv() return 0. self.handle_close() return '' And handle_close eventually ends up telling zope to close the connection. This has two possible fixes, but both are probably required. Firstly, it seems the latest python still has this problem, so I probably need to report it as a bug. Secondly, since zope still requires older python versions to run, a workaround probably needs to be found. I suppose just rebuilding the _socket native module will probably do it and I will test this as soon as I have more time. regards, Izak ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope closes connection if the client closes the write-end of connection
Hi all,
Yesterday we tried to get the backend probe feature of varnish to work
with zope. It failed, claiming that a response was not received.
Checking the Z2 log showed that the request was received but that zero
bytes was transferred. Using tcpdump shows that varnish closes the
connection before zope can send the response.
I eventually distilled the check varnish uses into a small C program,
and an interesting problem shows up. When you call shutdown(fd, SHUT_WR)
on your socket connection, in effect telling zope that you're done
talking to it, it looks like zope responds in kind by not talking to you
either. I deduce this from the fact that poll() returns POLLHUP in
revents and read() returns zero (EOF). POLLHUP, if I understand this
correctly, means the other side (zope) has hung up.
This effectively makes it impossible to use certain kinds of monitoring
software with zope, specifically varnish.
I would appreciate it if someone in the know could look at the code,
tell me what I (and by extension varnish) am doing wrong, or whether
this is perhaps a bug in zope (which I strongly suspect).
The attached C code works fine against apache, and it also works fine
against zope if you remove the shutdown() call.
regards,
Izak
--- snip: poll.c ---
#define URL "/"
#include
#include
#include
#include
#include
#include
#include
#include
#include
int main(int argc, char **argv){
struct pollfd fds[1];
int timeout_msecs = 5000;
int ret;
int i, sd, request_size, offset;
struct sockaddr_in addr;
char buf[1024];
if(argc < 3){
fprintf(stderr, "Usage: %s ip port\n", argv[0]);
return 1;
}
/* Open network connection */
sd = socket(AF_INET, SOCK_STREAM, 0);
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = inet_addr(argv[1]);
addr.sin_port = htons(atoi(argv[2]));
if (connect(sd, (struct sockaddr*)&addr, sizeof(addr)) != 0){
perror("connect");
return 1;
}
/* Send the request */
request_size = snprintf(buf, sizeof(buf),
"GET %s HTTP/1.0\r\nConnection: close\r\n\r\n", URL);
offset = 0;
while(offset < request_size){
i = write(sd, buf+offset, request_size-offset);
offset += i;
}
/* Shutdown the write side */
shutdown(sd, SHUT_WR);
/* Now wait for data to come back */
do {
fds[0].fd = sd;
fds[0].events = POLLIN;
fds[0].revents = 0;
ret = poll(fds, 1, timeout_msecs);
if (ret > 0) {
printf("%d POLLIN events on fd %d\n", ret, fds[0].fd);
printf("revents = %d\n", fds[0].revents);
i = read(sd, buf, sizeof(buf));
printf("Read %d bytes from %d\n", i, sd);
} else if (ret == 0){
printf("Timeout\n");
} else {
perror("poll");
}
} while(i>0);
}
--- snip ---
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
