#39809 [Fbk-Opn]: FastCGI Requests silently dropped
ID: 39809
User updated by: e at osterman dot com
Reported By: e at osterman dot com
-Status: Feedback
+Status: Open
Bug Type: CGI related
Operating System: FC6
PHP Version: 5.2.0
Assigned To: dmitry
New Comment:
Dimitry,
You're example shocked me. I tried something similar making a simple
stream_socket_server, but didn't make the client in PHP.
?
$server = stream_socket_server('tcp://127.0.0.1:1234');
if (!$server) {
die('Unable to create AF_INET socket');
}
sleep(10);
?
# telnet localhost 1234
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
I had expected the same behavior in PHP's fsockopen, but indeed, as you
said fsocketopen returns a valid resource even though the connection has
not yet been accept()'d by the server. More over, feof returns false and
fwrite to the socket returns the correct number of bytes written.
stream_get_metadata returns nothing interesting either.
How is a PHP fsockopen/stream_socket_client client to know when a
connection truely has been accept()'d?
The reason why this is all so important to us, is that we have a
cluster of frontend servers that use a pool of FCGI backend servers. We
need the busier frontend servers to dynamically open up more persistent
connections (FCGI_KEEP_CONN) to FCGI servers (and release them as
demand subsides). When a particular FCGI server is at capacity
(PHP_FCGI_CHILDREN), we need the client to try (in a round robin
fashion) another FCGI server. The current problem, as you understand by
now, is that the client get's stuck when connecting to an FCGI server at
capacity and timeout.
As you've now suggested, it's apparently b/c they getting stuck waiting
for the accept(), but never get it.
What recourse does the client have?
Regards,
Erik Osterman
Previous Comments:
[2006-12-18 13:15:46] [EMAIL PROTECTED]
No you are :)
But you made a great job writting test script, and I think this
discussion may lead us to some good result.
The fact that connect() returns file descriptor doesn't mean that
accept() was really called. See the folllowing script. You can even
write to socket befor it is really accepted.
?php
$server = stream_socket_server('tcp://127.0.0.1:1234');
if (!$server) {
die('Unable to create AF_INET socket [server]');
}
$socket1 = fsockopen('127.0.0.1', 1234, $errno, $errstr, 5);
if (!$socket1)
die(Failed to connect to 127.0.0.1:1234\n);
fwrite($socket1, Hello\n);
$socket2 = fsockopen('127.0.0.1', 1234, $errno, $errstr, 5);
if (!$socket2)
die(Failed to connect to 127.0.0.1:1234\n);
fwrite($socket2, World\n);
$socket = stream_socket_accept($server);
$data = fgets($socket, 1024);
echo($data);
fclose($socket);
$socket = stream_socket_accept($server);
$data = fgets($socket, 1024);
echo($data);
fclose($socket);
fclose($socket2);
fclose($socket1);
fclose($server);
?
[2006-12-15 21:35:04] e at osterman dot com
PHP process DOESN'T accept() new connections
if it already has persistent connection opened.
Note that php/fastcgi is one-process-one-connection
server that doesn't implement multiplexion
If this were actually the case, I'd be satisfied. That would mean the
FCGI clients would get connection refused when there are no more
sockets/children available. But what actually happens is that the
connection is established, meaning accept() does get called.
To test this, modify the example like this
// Open up the first connection
$socket1 = FCGI_Connect('localhost', 1234);
// Send a request with FCGI_KEEP_CONN
FCGI_Test($socket1);
// Open up the second connection (should be refused)
$socket2 = FCGI_Connect('localhost', 1234);
printf(socket1:%d socket2:%d\n, feof($socket1), feof($socket2));
Expected output:
socket1:0 socket2:1
Actual output:
socket1:0 socket2:0
In otherwords, both connections are established = accept() was
called.
Am I making sense?
Regards,
Erik Osterman
[2006-12-15 15:00:55] [EMAIL PROTECTED]
it simply accepts/ignores them
PHP process DOESN'T accept() new connections if it already has
persistent connection opened. Note that php/fastcgi is
one-process-one-connection server that doesn't implement multiplexion
(like apache 1.3).
PHP doesn't try to manage persistent connection itself, however FastCGI
module may do it (especially in multithreaded environment).
[2006-12-13 21:26:46] e at osterman dot com
I know if you open up 1 socket to one child, it works:
If you only open
#39809 [Fbk-Opn]: FastCGI Requests silently dropped
ID: 39809
User updated by: e at osterman dot com
Reported By: e at osterman dot com
-Status: Feedback
+Status: Open
Bug Type: CGI related
Operating System: FC6
PHP Version: 5.2.0
Assigned To: dmitry
New Comment:
By changing the example code to use PHP's sockets (e.g.
socket_create/connect/read/write) allows one to use the socket_shutdown
as you suggested. But calling shutdown after the request defeats the
purpose of persistent connections, since that just negates the effect
of passing FCGI_KEEP_CONN. Connections cannot be reused to send
requests once they're shutdown after a request.
So, it sounds like PHP's design decision is to support PHP_KEEP_CONN,
but when PHP_FCGI_CHILDREN is reached, all new connections are
queued/blocked. When a client releases its connection to a php-cgi
child process, then the next connection in the queue is handed over to
the next available child. While this makes a lot of sense when you're
dealing with only non-persistent connections or only a single php-cgi
server, it's not desireable when you have lots of persistent
connections or lots of fcgi servers. For example, in HTTP/1.1, servers
respond 503 Server too busy. If the client were notified in some way,
either by way of a closed connection, refused connection, or
FCGI_OVERLOADED packet, then the client could try another FCGI server.
But since that doesn't happen, the client must just sit and wait.
Perhaps, indefinitely or just timeout =(
What is your take on this?
Best Regards,
Erik Osterman
Previous Comments:
[2006-12-18 18:53:58] [EMAIL PROTECTED]
?
$socket1 = FCGI_Connect('localhost', 1234);
FCGI_Test($socket1);
FCGI_Response($socket1);
sleep(30);
fclose($socket1);
?
this code is not perferct, and this is the reason of bad behavior. You
should call shutdown() man 2 shutdown after writing request into
socket, but PHP doesn't implement such function.
?
$socket1 = FCGI_Connect('localhost', 1234);
FCGI_Test($socket1);
shutdown($socket1, SHUT_WR); //we don't have this function
FCGI_Response($socket1);
sleep(30);
fclose($socket1);
?
I cannot repeat your behavior with telnet. I always get Connection
closed after a timeout.
BTW you can insert debug output into fastcgi.c right after accept() and
try it.
[2006-12-18 18:29:13] e at osterman dot com
For what it's worth, this shows some conflicting data. Perhaps I'm just
interpretting it wrong.
Start theh FCGI server in single process mode
# php-cgi -b 1234
Modify the test script as follows:
?
$socket1 = FCGI_Connect('localhost', 1234);
FCGI_Test($socket1);
FCGI_Response($socket1);
sleep(30);
fclose($socket1);
?
Now, the single process php-cgi server should not be accept()ing any
more connections for 30 seconds.
# telnet localhost 1234
Trying 127.0.0.1...
Connected to localhost.localdomain.
Escape character is '^]'.
Per my previous post, when a stream_socket_server server does not call
accept(), the 'telnet' session gets connection refused. In this
example, it's not getting refused by the php-cgi server, which leads me
to believe that php-cgi might actually be calling accept. This could be
a implementation difference.. but I don't know.
Regards,
Erik Osterman
[2006-12-18 18:20:11] e at osterman dot com
Dimitry,
You're example shocked me. I tried something similar making a simple
stream_socket_server, but didn't make the client in PHP.
?
$server = stream_socket_server('tcp://127.0.0.1:1234');
if (!$server) {
die('Unable to create AF_INET socket');
}
sleep(10);
?
# telnet localhost 1234
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
I had expected the same behavior in PHP's fsockopen, but indeed, as you
said fsocketopen returns a valid resource even though the connection has
not yet been accept()'d by the server. More over, feof returns false and
fwrite to the socket returns the correct number of bytes written.
stream_get_metadata returns nothing interesting either.
How is a PHP fsockopen/stream_socket_client client to know when a
connection truely has been accept()'d?
The reason why this is all so important to us, is that we have a
cluster of frontend servers that use a pool of FCGI backend servers. We
need the busier frontend servers to dynamically open up more persistent
connections (FCGI_KEEP_CONN) to FCGI servers (and release them as
demand subsides). When a particular FCGI server is at capacity
(PHP_FCGI_CHILDREN), we need the client to try (in a round robin
fashion) another FCGI server. The current problem, as you understand by
now, is that the client get's stuck when connecting to an FCGI server at
capacity and timeout.
As you've now suggested, it's apparently b/c they getting stuck waiting
for the
#39809 [Fbk-Opn]: FastCGI Requests silently dropped
ID: 39809
User updated by: e at osterman dot com
Reported By: e at osterman dot com
-Status: Feedback
+Status: Open
Bug Type: CGI related
Operating System: FC6
PHP Version: 5.2.0
Assigned To: dmitry
New Comment:
PHP process DOESN'T accept() new connections
if it already has persistent connection opened.
Note that php/fastcgi is one-process-one-connection
server that doesn't implement multiplexion
If this were actually the case, I'd be satisfied. That would mean the
FCGI clients would get connection refused when there are no more
sockets/children available. But what actually happens is that the
connection is established, meaning accept() does get called.
To test this, modify the example like this
// Open up the first connection
$socket1 = FCGI_Connect('localhost', 1234);
// Send a request with FCGI_KEEP_CONN
FCGI_Test($socket1);
// Open up the second connection (should be refused)
$socket2 = FCGI_Connect('localhost', 1234);
printf(socket1:%d socket2:%d\n, feof($socket1), feof($socket2));
Expected output:
socket1:0 socket2:1
Actual output:
socket1:0 socket2:0
In otherwords, both connections are established = accept() was
called.
Am I making sense?
Regards,
Erik Osterman
Previous Comments:
[2006-12-15 15:00:55] [EMAIL PROTECTED]
it simply accepts/ignores them
PHP process DOESN'T accept() new connections if it already has
persistent connection opened. Note that php/fastcgi is
one-process-one-connection server that doesn't implement multiplexion
(like apache 1.3).
PHP doesn't try to manage persistent connection itself, however FastCGI
module may do it (especially in multithreaded environment).
[2006-12-13 21:26:46] e at osterman dot com
I know if you open up 1 socket to one child, it works:
If you only open up 1 socket, and run multiple requests it works
fine.
That's not the bug. The bug is PHP doesn't handle persistent
connections (FCGI_KEEP_CONN), when the number of persistent connections
exceedes the number of php children. The fcgi spec states that if the
application doesn't have enough resoures to complete the request (e.g
database handles, or in the case of PHP enough children), that it
should return that it's overloaded. PHP does not do this; it simply
accepts/ignores them. What PHP does is rely on the connection queueing,
which doesn't solve the KEEP_CONN problem. Constantly opening up
connections is inefficient.
Regards,
Erik Osterman
[2006-12-13 14:44:51] [EMAIL PROTECTED]
In your example you use persistent FastCGI connections
(FCGI_KEEP_CONN). It means web server connects to PHP and sends SEVERAL
requests using the SAME socket then it can close connection. You can
correct your example in the following way to use persistent conection:
$socket1 = FCGI_Connect('localhost', 1234);
FCGI_Test($socket1);
FCGI_Response($socket1);
FCGI_Test($socket1);
FCGI_Response($socket1);
or you may not to use persistent connection and then you must close
connection
$socket1 = FCGI_Connect('localhost', 1234);
FCGI_Test($socket1);
FCGI_Response($socket1);
fclose($socket1);
$socket2 = FCGI_Connect('localhost', 1234);
FCGI_Test($socket2);
FCGI_Response($socket2);
fclose($socket2);
In case of non-persistent connection usgage of shutdown() right after
sending request is much better then close() after reading response.
[2006-12-13 06:55:06] e at osterman dot com
Reproduce Code:
?
// test-fcgi.php - a sample FCGI client
define('FCGI_VERSION_1', 1);
define('FCGI_BEGIN_REQUEST', 1);
define('FCGI_ABORT_REQUEST', 2);
define('FCGI_END_REQUEST', 3);
define('FCGI_PARAMS', 4);
define('FCGI_STDIN', 5);
define('FCGI_STDOUT', 6);
define('FCGI_STDERR', 7);
define('FCGI_DATA', 8);
define('FCGI_GET_VALUES', 9);
define('FCGI_GET_VALUES_RESULT', 10);
define('FCGI_RESPONDER', 1);
define('FCGI_KEEP_CONN', 1);
function FCGI_Packet($type, $content)
{
$len=strlen($content);
$packet=chr(FCGI_VERSION_1).chr($type).chr(0).chr(1).chr((int)($len/256)).chr($len%256).chr(0).chr(0).$content;
return($packet);
}
function FCGI_NVPair($name, $value)
{
$nlen = strlen($name);
$vlen = strlen($value);
if ($nlen 128)
$nvpair = chr($nlen);
else
$nvpair = chr(($nlen 24) | 0x80) . chr(($nlen 16) 0xFF) .
chr(($nlen 8) 0xFF) . chr($nlen 0xFF);
if ($vlen 128)
$nvpair .= chr($vlen);
else
$nvpair .= chr(($vlen 24) | 0x80) . chr(($vlen 16) 0xFF) .
chr(($vlen 8) 0xFF) . chr($vlen 0xFF);
return $nvpair . $name . $value;
}
function FCGI_Decode($data)
{
if( strlen($data) 8 )
die(Packet too small . strlen($data) . \n);
$length = (ord($data{4}) 8)+ord($data{5});
$packet = Array( 'version' = ord($data{0}),
