Req #35326 [Com]: Concurrency issue with recursive mkdir
Edit report at https://bugs.php.net/bug.php?id=35326edit=1 ID: 35326 Comment by: gustavo dot straube at gmail dot com Reported by:bugs dot php dot net at chsc dot dk Summary:Concurrency issue with recursive mkdir Status: Not a bug Type: Feature/Change Request Package:*General Issues Operating System: Linux PHP Version:5.1.6 Block user comment: N Private report: N New Comment: We don't need a locking mechanism. If the creation of a directory fails, we just need to check if it exists anyway [...] In the application I experienced this bug, I do have a condition before the directory creation. 1. if (!is_dir($path)) 2. mkdir($path, 0755, true); If I have two concurrent process (A and B) running the code in this order (process - line): A - 1 B - 1 B - 2 A - 2 When process A verified the directory existence, it really didn't exist. Previous Comments: [2013-10-30 08:49:24] bugs dot php dot net at chsc dot dk To create the right locking mechanism to avoid such issue to happen We don't need a locking mechanism. If the creation of a directory fails, we just need to check if it exists anyway (i.e. it has been created by a concurrent thread). If it does, we can proceed to create the subdirectories. [2013-10-01 04:52:05] paj...@php.net Update the status. And adding a small note about why is it not a php bug, what is described here can happen anywhere outside php as well, be a shell script, ftp/ssh, etc. [2013-10-01 04:21:25] paj...@php.net Should have been: ... and will drastically affect performance without actually solving the problem [2013-10-01 04:15:57] paj...@php.net I am not sure it is actually a PHP problem. To create the right locking mechanism to avoid such issue to happen between (almost) simultaneous requests is the application developer job, not the core. Adding such tests should be done in almost all file creation operations (and some other) and will drastically performance without actually solving the problem. [2013-09-30 10:39:45] me at vlastv dot ru Problem still exists in PHP 5.4.1. The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=35326 -- Edit this bug report at https://bugs.php.net/bug.php?id=35326edit=1
Req #35326 [Com]: Concurrency issue with recursive mkdir
Edit report at https://bugs.php.net/bug.php?id=35326edit=1
ID: 35326
Comment by: me at vlastv dot ru
Reported by:bugs dot php dot net at chsc dot dk
Summary:Concurrency issue with recursive mkdir
Status: Open
Type: Feature/Change Request
Package:Feature/Change Request
Operating System: Linux
PHP Version:5.1.6
Block user comment: N
Private report: N
New Comment:
Problem still exists in PHP 5.4.1.
Previous Comments:
[2013-07-23 22:55:07] gustavo dot straube at gmail dot com
Problem still exists in PHP 5.3.3.
[2006-10-10 07:25:52] bugs dot php dot net at chsc dot dk
Problem still exists in PHP 5.1.6.
[2005-11-22 13:52:24] bugs dot php dot net at chsc dot dk
Yes, it also occurs with the CVS snapshot (this time I tried with the CGI
version).
AFAICS the problem is in php_plain_files_mkdir in main/streams/plain_wrapper.c.
If php_mkdir() or VCWD_MKDIR() returns false, the method gives up instead of
checking whether directory exists anyway, i.e. if it was created by another
thread.
[2005-11-22 12:13:10] tony2...@php.net
Please try using this CVS snapshot:
http://snaps.php.net/php5-latest.tar.gz
For Windows:
http://snaps.php.net/win32/php5-win32-latest.zip
[2005-11-22 11:32:49] bugs dot php dot net at chsc dot dk
Description:
There appears to be a concurrency issue when mkdir is called on two separate
threads with the following parameters:
Thread 1:
mkdir('/tmp/foo/bar/1', 0777, true);
Thread 2:
mkdir('/tmp/foo/bar/2', 0777, true);
Sometimes one of the threads (e.g. thread 2) two fails with Warning: mkdir():
File exists and the directory /tmp/foo/bar/2 does not exist afterwards.
What I think is happening in thread 2 is is this:
PHP looks for /tmp/foo/bar/2 and sees that it does not exist. Neither does
/tmp/foo/bar and /tmp/foo. So it tries to create /tmp/foo, but that directory
was just created a millisecond second ago by another thread, so mkdir gives
instead of just moving on to create /tmp/foo/bar and /tmp/foo/bar/2.
Reproduce code:
---
This code reproduces the problem in about every second attempt. It consists of
two files, index.php and iframe.php.
index.php:
?php
$fire = time() + 2;
?
h1Nr. 1/h1
iframe src=iframe.php?fire=?= $fire ?amp;no=1/iframe
h1Nr. 2/h1
iframe src=iframe.php?fire=?= $fire ?amp;no=2/iframe
iframe.php:
?php
$fire = $_GET['fire'];
$no = $_GET['no'];
// synchronize the two threads
while ($fire time());
$dir = '/tmp/' . $fire . '/foo/bar/foo/bar/' . $no;
//sleep($no);
var_dump(is_dir($dir));
echo Making $dir ;
mkdir($dir, 0777, true);
var_dump(is_dir($dir));
?
Expected result:
Thread 2 should output this:
bool(false) Making /tmp/1132653651/foo/bar/foo/bar/2 bool(true)
Actual result:
--
Thread 2 sometimes outputs this:
bool(false) Making /tmp/1132655181/foo/bar/foo/bar/2 Warning: mkdir(): File
exists in /home/chsc/public_html/mkdir/iframe.php on line 10 bool(false)
Notice that the directory does not exist afterwards.
If I uncomment the sleep() call above, there are no problems.
A workaround:
$ok = mkdir($dir, 0777, true);
if (!$ok !is_dir($dir)) {
sleep(1);
mkdir($dir, 0777, true);
}
In other words, it works if I try again. I hope that mkdir() can be changed so
that it only returns false if the directory cannot be created at all (e.g.
because of lack of permissions or because it already exists).
--
Edit this bug report at https://bugs.php.net/bug.php?id=35326edit=1
Req #35326 [Com]: Concurrency issue with recursive mkdir
Edit report at https://bugs.php.net/bug.php?id=35326edit=1
ID: 35326
Comment by: gustavo dot straube at gmail dot com
Reported by:bugs dot php dot net at chsc dot dk
Summary:Concurrency issue with recursive mkdir
Status: Open
Type: Feature/Change Request
Package:Feature/Change Request
Operating System: Linux
PHP Version:5.1.6
Block user comment: N
Private report: N
New Comment:
Problem still exists in PHP 5.3.3.
Previous Comments:
[2006-10-10 07:25:52] bugs dot php dot net at chsc dot dk
Problem still exists in PHP 5.1.6.
[2005-11-22 13:52:24] bugs dot php dot net at chsc dot dk
Yes, it also occurs with the CVS snapshot (this time I tried with the CGI
version).
AFAICS the problem is in php_plain_files_mkdir in main/streams/plain_wrapper.c.
If php_mkdir() or VCWD_MKDIR() returns false, the method gives up instead of
checking whether directory exists anyway, i.e. if it was created by another
thread.
[2005-11-22 12:13:10] tony2...@php.net
Please try using this CVS snapshot:
http://snaps.php.net/php5-latest.tar.gz
For Windows:
http://snaps.php.net/win32/php5-win32-latest.zip
[2005-11-22 11:32:49] bugs dot php dot net at chsc dot dk
Description:
There appears to be a concurrency issue when mkdir is called on two separate
threads with the following parameters:
Thread 1:
mkdir('/tmp/foo/bar/1', 0777, true);
Thread 2:
mkdir('/tmp/foo/bar/2', 0777, true);
Sometimes one of the threads (e.g. thread 2) two fails with Warning: mkdir():
File exists and the directory /tmp/foo/bar/2 does not exist afterwards.
What I think is happening in thread 2 is is this:
PHP looks for /tmp/foo/bar/2 and sees that it does not exist. Neither does
/tmp/foo/bar and /tmp/foo. So it tries to create /tmp/foo, but that directory
was just created a millisecond second ago by another thread, so mkdir gives
instead of just moving on to create /tmp/foo/bar and /tmp/foo/bar/2.
Reproduce code:
---
This code reproduces the problem in about every second attempt. It consists of
two files, index.php and iframe.php.
index.php:
?php
$fire = time() + 2;
?
h1Nr. 1/h1
iframe src=iframe.php?fire=?= $fire ?amp;no=1/iframe
h1Nr. 2/h1
iframe src=iframe.php?fire=?= $fire ?amp;no=2/iframe
iframe.php:
?php
$fire = $_GET['fire'];
$no = $_GET['no'];
// synchronize the two threads
while ($fire time());
$dir = '/tmp/' . $fire . '/foo/bar/foo/bar/' . $no;
//sleep($no);
var_dump(is_dir($dir));
echo Making $dir ;
mkdir($dir, 0777, true);
var_dump(is_dir($dir));
?
Expected result:
Thread 2 should output this:
bool(false) Making /tmp/1132653651/foo/bar/foo/bar/2 bool(true)
Actual result:
--
Thread 2 sometimes outputs this:
bool(false) Making /tmp/1132655181/foo/bar/foo/bar/2 Warning: mkdir(): File
exists in /home/chsc/public_html/mkdir/iframe.php on line 10 bool(false)
Notice that the directory does not exist afterwards.
If I uncomment the sleep() call above, there are no problems.
A workaround:
$ok = mkdir($dir, 0777, true);
if (!$ok !is_dir($dir)) {
sleep(1);
mkdir($dir, 0777, true);
}
In other words, it works if I try again. I hope that mkdir() can be changed so
that it only returns false if the directory cannot be created at all (e.g.
because of lack of permissions or because it already exists).
--
Edit this bug report at https://bugs.php.net/bug.php?id=35326edit=1
