Edit report at https://bugs.php.net/bug.php?id=55438&edit=1
ID: 55438
Comment by: phpnet at lostreality dot org
Reported by: xuefer at gmail dot com
Summary: race condition: curlwapper is not sending http
header randomly
Status: Assigned
Type: Bug
Package: cURL related
Operating System: gentoo
PHP Version: 5.3.6
Assigned To: pierrick
Block user comment: N
Private report: N
New Comment:
I have curl-7.15.5-15.el5 according to rpm -q, but I can only locate
/usr/lib/libcurl.so.3.0.0 and /usr/lib64/libcurl.so.3.0.0 on my machine I'm
testing on (CentOS 5.8). The binary says: /usr/bin/curl -V
curl 7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3
libidn/0.6.5
Normally, I use the stock RPMs for PHP, but a recent project I was working on
failed to run properly on another machine, where the owners also use CentOS
5.8, but use CPanel/WHM instead of the CentOS RPMs for PHP. CPanel uses the
--with-curlwrappers option, where as the stock CentOS and RHEL RPMs have never
used that option on any of their builds. It took a lot of digging before I
realized that it was the --with-curlwrappers option that caused the scripts to
fail on that machine while working perfectly on mine.
To verify if the headers were actually sent, I used: tcpdump -i eth1 -Als0 host
www.example.com
I had two PuTTY windows open, one with tcpdump, the other running the test
script I mentioned before with: ./php-5.4.9/sapi/cli/php ./test.php
It was pretty clear to me that the headers were never sent before the patch,
and always sent after the patch.
Previous Comments:
------------------------------------------------------------------------
[2012-12-19 05:50:16] pierr...@php.net
I tried to reproduce this bug but wasn't able to do it.
Could you give me more details on the libcurl version used by your PHP instance
?
And also, how do you make sure that the headers are not properly sent ?
------------------------------------------------------------------------
[2012-12-19 04:33:11] phpnet at lostreality dot org
I submitted a patch that moves the slist from a local variable in
php_curl_stream_opener() into the php_curl_stream struct. The headers are no
longer cleared and freed at the end of php_curl_stream_opener(). The code to
free the slist is moved into php_curl_stream_close() instead. I'm not sure if
this is the best approach, but it clearly gives me a 100% success rate with
having headers get sent, where as I had a literal 0% success rate before (not
sure if there is really a race condition or not, just that the headers get
cleared and the slist freed before they get used.)
The test code I used was as follows (Actual cookie and URL redacted)
<?php
$opt = array('http' => array('method' => 'GET', 'header' => 'Cookie: foo=bar'));
$ctx = stream_context_create($opt);
$f = fopen('http://www.example.com/', 'r', false, $ctx);
fread($f, 1); //work-around curl-wrappers bug where meta_data doesn't exist
until the stream is read
$data = stream_get_meta_data($f);
fclose($f);
var_dump($data);
?>
I compiled PHP with the following flags (not that I think anything matters to
this bug other than --with-curlwrappers):
--enable-static --with-mcrypt --with-ldap --with-iconv --enable-mbstring
--with-gd --enable-mbregex --with-zlib --with-imap --enable-ftp --with-gettext
--enable-sockets --with-mysql=/usr --enable-cgi --with-imap-ssl
--enable-sockets --with-pdo-mysql --with-openssl --with-kerberos --with-curl
--with-curlwrappers --with-tidy --with-pcre-regex --with-bz2 --enable-zip
--with-libdir=/lib64
------------------------------------------------------------------------
[2012-12-18 19:29:56] phpnet at lostreality dot org
I think I am seeing this same problem too (On 5.3.10, but nothing has changed
in the source in 5.4.9 either).
Can you explain how this is happening, or suggest a work-around? I was digging
into the PHP source, expecting that the --with-curlwrappers option was
basically broken and incomplete. I was surprised to find the line:
curl_easy_setopt(curlstream->curl, CURLOPT_HTTPHEADER, slist);
Because that code all seems to indicate that the headers should be sent, but no
matter what I try, nothing I put in headers ever appears in the actual request.
I keep running tcpdump but I never see the headers I put in http->header.
------------------------------------------------------------------------
[2011-08-17 11:45:54] xuefer at gmail dot com
sorry for the mismatch http url string. i was trying to remove some string for
privacy
------------------------------------------------------------------------
[2011-08-17 11:38:50] xuefer at gmail dot com
Description:
------------
background: php is configured with curl wrapper, which make file_get_contents
use curl. i haven't tested with calling curl functions directly
php unset curl header too soon before curl make the request
expected order: set header, build and send request. unset header
actual order 1: set header, build and send request. unset header (good)
actual order 2: set header, unset header, build and send request (bad)
"send request" comes after php "unset header"
curl behavior randomly, by sending request before or after php unset the header
Test script:
---------------
#!/usr/lib/php5.3/bin/php
<?php
for (;;) {
$username = 'test1';
$password = mt_rand(0, 99999) .
'....................................................................................';
$authUrl = "http://localhost/";;
$context = stream_context_create(array(
'http' => array(
'header' => "Authorization: Basic " .
base64_encode("$username:$password")
)
));
$http_response_header = array();
$data = file_get_contents($authUrl, false, $context);
sleep(1);
}
?>
tcpdump -nilo dst port 80 -w- -s0
Expected result:
----------------
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
Authorization: Basic
dGVzdDE6MTQzNjEuLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4u
Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4=
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
Authorization: Basic
dGVzdDE6NTY3MDQuLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4u
Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4=
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
Authorization: Basic
dGVzdDE6MTQzNjEuLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4u
Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4=
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
Authorization: Basic
dGVzdDE6NTY3MDQuLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4u
Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4=
Actual result:
--------------
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
Authorization: Basic
dGVzdDE6MTQzNjEuLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4u
Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4=
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
Authorization: Basic
dGVzdDE6NTY3MDQuLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4u
Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4=
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
gdb --args /usr/lib/php5.3/bin/php.debug test.php
(gdb) l /usr/src/debug/dev-lang/php-5.3.6/sapis-build/cli/ext/curl/streams.c:367
360 slist =
curl_slist_append(slist,
trimmed);
361 efree(trimmed);
362 p = php_strtok_r(NULL, "\r\n",
&token);
363 }
364 efree(copy_ctx_opt);
365 }
366 if (slist) {
367 curl_easy_setopt(curlstream->curl,
CURLOPT_HTTPHEADER, slist);
368 }
369 }
370 if (SUCCESS == php_stream_context_get_option(context,
"http", "method", &ctx_opt) && Z_TYPE_PP(ctx_opt) == IS_STRING) {
371 if (strcasecmp(Z_STRVAL_PP(ctx_opt), "get")) {
372 if (!strcasecmp(Z_STRVAL_PP(ctx_opt),
"head")) {
373 curl_easy_setopt(curlstream-
>curl, CURLOPT_NOBODY, 1);
374 } else {
(gdb) l /usr/src/debug/dev-lang/php-5.3.6/sapis-build/cli/ext/curl/streams.c:501
494 if (msg_found) {
495 goto exit_fail;
496 }
497 }
498
499 /* context headers are not needed anymore */
500 if (slist) {
501 curl_easy_setopt(curlstream->curl, CURLOPT_HTTPHEADER,
NULL);
502 curl_slist_free_all(slist);
503 }
504 return stream;
505
506 exit_fail:
507 php_stream_close(stream);
508 if (slist) {
(gdb) br 367
Breakpoint 1 at 0x1c59c9: file /usr/src/debug/dev-lang/php-5.3.6/sapis-
build/cli/ext/curl/streams.c, line 367.
(gdb) br 501
Breakpoint 2 at 0x1c61bc: file /usr/src/debug/dev-lang/php-5.3.6/sapis-
build/cli/ext/curl/streams.c, line 501.
(gdb) br Curl_send_plain
Breakpoint 4 at 0xb7698ca0: file sendf.c, line 279.
(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /usr/lib/php5.3/bin/php ./test
[Thread debugging using libthread_db enabled]
warning: Lowest section in /usr/lib/libicudata.so.48 is .hash at 000000f4
[New Thread 0xb55e76f0 (LWP 31857)]
[Switching to Thread 0xb55e76f0 (LWP 31857)]
Breakpoint 1, php_curl_stream_opener (wrapper=0x80aacd44, filename=0x80c45c24
"http://localhost/vbb/authapi.php";,
mode=0x80a2a01b "rb", options=4, opened_path=0x0, context=0x80c47224)
at /usr/src/debug/dev-lang/php-5.3.6/sapis-build/cli/ext/curl/streams.c:367
367 curl_easy_setopt(curlstream->curl,
CURLOPT_HTTPHEADER, slist);
(gdb) c
Continuing.
Breakpoint 2, php_curl_stream_opener (wrapper=0x80aacd44, filename=0x80c45c24
"http://localhost/";,
mode=0x80a2a01b "rb", options=4, opened_path=0x0, context=0x80c47224)
at /usr/src/debug/dev-lang/php-5.3.6/sapis-build/cli/ext/curl/streams.c:501
501 curl_easy_setopt(curlstream->curl, CURLOPT_HTTPHEADER,
NULL);
(gdb) c
Continuing.
Breakpoint 4, Curl_send_plain (conn=0x80cd0628, num=0, mem=0x80cd1120, len=82,
code=0xbff162f4) at sendf.c:279
279 {
(gdb) p ((char*)mem)[0]@len
$1 = "GET /vbb/authapi.php HTTP/1.1\r\nUser-Agent:
PHP/5.3.6-pl0-gentoo\r\nHost:
localhost\r\nAccept: */*\r\n\r\n"
Authorization header is not there
===========================
to reproduce this bug, curl is built with ares and without threads
(in another case http request is not even send, when curl is built with threads
and without ares, was a curl issue i guess)
# equery uses net-misc/curl
[ Legend : U - final flag setting for installation]
[ : I - package is installed with flag ]
[ Colors : set, unset ]
* Found these USE flags for net-misc/curl-7.21.4:
U I
+ + ares : Enabled c-ares dns support
+ + gnutls : Prefer gnutls over nss and openssl as the crypto engine
- - idn : Enable support for Internationalized Domain Names
+ + ipv6 : Adds support for IP version 6
+ + kerberos : Adds kerberos support
- - ldap : Adds LDAP support (Lightweight Directory Access Protocol)
- - libssh2 : Enabled SSH urls in curl using libssh2
- - nss : Prefer NSS over openssl as the crypto engine
+ + ssl : Enable crypto engine support (via openssl if USE='-gnutls -
nss')
- - static-libs : Build static libraries
- - test : Workaround to pull in packages needed to run with
FEATURES=test. Portage-2.1.2 handles this internally,
so don't set it in make.conf/package.use anymore
- - threads : Adds threads support for various packages. Usually pthreads
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=55438&edit=1
