From: k at ailis dot de
Operating system: Linux
PHP version: 4.3.8
PHP Bug Type: Performance problem
Bug description: Many uploads will slow down Linux machine
Description:
------------
This is a mix of "performance problem" and "feature request". So feel free
to reclassify this report as "Feature request" if you think it fits
better.
In a large project we have the problem that many users are uploading many
large files to our PHP server. PHP can handle this very well but the
problem is, that all files are stored in the same directory. Our upload
directory sometimes contains more than 20000 files. This produces high
loads because accessing these files is very I/O intensive because of the
flat directory hierarchy.
You already have addressed this problem with session files by introducing
a "directory depth". Its possible to use "2;/var/lib/php/uploads" as
session.save_path which is a very nice feature. Exactly this feature would
be a cool thing if it were also implemented for "upload_tmp_dir".
Here is a small patch as an example how this feature could be implemented.
We are using this patch on our servers and it works great. This patch not
only implements this feature for upload_tmp_dir but for all other modules
which are using the php_open_temporary_file() function.
--- php_open_temporary_file.c.orig Sun Aug 8 14:29:06 2004
+++ php_open_temporary_file.c Sun Aug 8 14:29:12 2004
@@ -106,6 +106,10 @@
{
char *trailing_slash;
char *opened_path;
+ char *p;
+ char *sub_dirs;
+ size_t dirdepth, i;
+ int rand_value;
int fd = -1;
#ifndef HAVE_MKSTEMP
int open_flags = O_CREAT | O_TRUNC | O_RDWR
@@ -125,6 +129,30 @@
if (!(opened_path = emalloc(MAXPATHLEN))) {
return -1;
}
+
+ /* Check subdirectory depth and point p to real path name */
+ if ((p = strchr(path, ';')))
+ {
+ dirdepth = (size_t) strtol(path, NULL, 10);
+ p++;
+ }
+ else
+ {
+ dirdepth = 0;
+ p = (char *) path;
+ }
+
+ /* Build subdirectory tree if needed */
+ if (dirdepth) {
+ if (!(sub_dirs = emalloc(dirdepth * 2 + 1))) return -1;
+ sub_dirs[0] = 0;
+ for (i = 0; i < dirdepth; i++)
+ {
+ strcat(sub_dirs, "x/");
+ rand_value = php_rand() & 0xf;
+ sub_dirs[i * 2] = rand_value < 10 ? rand_value + 48 : rand_value
+ 87;
+ }
+ } else sub_dirs = "";
if (IS_SLASH(path[strlen(path)-1])) {
trailing_slash = "";
@@ -132,7 +160,10 @@
trailing_slash = "/";
}
- (void)snprintf(opened_path, MAXPATHLEN, "%s%s%sXXXXXX", path,
trailing_slash, pfx);
+ (void)snprintf(opened_path, MAXPATHLEN, "%s%s%s%sXXXXXX", p,
trailing_slash, sub_dirs, pfx);
+
+ /* Free memory if it was used */
+ if (dirdepth) efree(sub_dirs);
#ifdef PHP_WIN32
if (GetTempFileName(path, pfx, 0, opened_path)) {
@@ -140,7 +171,7 @@
* which means that opening it will fail... */
VCWD_CHMOD(opened_path, 0600);
fd = VCWD_OPEN_MODE(opened_path, open_flags, 0600);
- }
+l1 }
#elif defined(NETWARE)
/* Using standard mktemp() implementation for NetWare */
file_path = mktemp(opened_path);
--
Edit bug report at http://bugs.php.net/?id=29571&edit=1
--
Try a CVS snapshot (php4): http://bugs.php.net/fix.php?id=29571&r=trysnapshot4
Try a CVS snapshot (php5): http://bugs.php.net/fix.php?id=29571&r=trysnapshot5
Fixed in CVS: http://bugs.php.net/fix.php?id=29571&r=fixedcvs
Fixed in release: http://bugs.php.net/fix.php?id=29571&r=alreadyfixed
Need backtrace: http://bugs.php.net/fix.php?id=29571&r=needtrace
Need Reproduce Script: http://bugs.php.net/fix.php?id=29571&r=needscript
Try newer version: http://bugs.php.net/fix.php?id=29571&r=oldversion
Not developer issue: http://bugs.php.net/fix.php?id=29571&r=support
Expected behavior: http://bugs.php.net/fix.php?id=29571&r=notwrong
Not enough info: http://bugs.php.net/fix.php?id=29571&r=notenoughinfo
Submitted twice: http://bugs.php.net/fix.php?id=29571&r=submittedtwice
register_globals: http://bugs.php.net/fix.php?id=29571&r=globals
PHP 3 support discontinued: http://bugs.php.net/fix.php?id=29571&r=php3
Daylight Savings: http://bugs.php.net/fix.php?id=29571&r=dst
IIS Stability: http://bugs.php.net/fix.php?id=29571&r=isapi
Install GNU Sed: http://bugs.php.net/fix.php?id=29571&r=gnused
Floating point limitations: http://bugs.php.net/fix.php?id=29571&r=float
