ID: 40261
Comment by: bloire at citytoo dot com
Reported By: thuejk at gmail dot com
Status: Assigned
Bug Type: Performance problem
Operating System: All
PHP Version: 5.2.0
Assigned To: dmitry
New Comment:
I have exactly the same probleme with foreach with big rows from mysql
and reaffection for each field for each row
After that, all processing is verry verry verry VERRY slow. It's
HORRIBLE. It's verry strange because I didn"t have this probleme with
php < 5.2
I hope that will be repair because now, I'm not trusting with php 5.2
Thank you for all
Previous Comments:
------------------------------------------------------------------------
[2007-03-16 12:09:04] thuejk at gmail dot com
I think I am hitting this in practice, or something like it.
I have a function call
<?
function lala() {
[database access with lots of data]
echo "returning " . time();
return;
}
lala();
echo "returned ".time();
?>
And I can see that for some reason, the time between "returning" and
"returned" is 60 seconds! This only happens the first time this
function is called, for some reason. Installing php 5.1.6 it returns
instantaneously.
I liked thet PHP 5.1 memory allocator better :(. The PHP 5.1 memory
allocator was also 1/4 the size of the PHP 5.2 memory allocator.
------------------------------------------------------------------------
[2007-01-30 14:53:40] thuejk at gmail dot com
The latest PHP snapshot does fix my example, and probably makes my
production code work.
This will probably fix the problem in far most cases. But it is
possible to construct an example which still have the problematic
behavior. One example is below.
<?php
$num = 100000;
$a = Array();
for ($i=0; $i<$num; $i++) {
$a[$i] = Array(1);
}
for ($i=0; $i<$num; $i++) {
$b[$i] = $a[$i][0];
}
unset($a);
for ($i=0; $i<$num; $i++) {
$b[$i] =
"1234567890123456789012345678901234567890123456789012345678901234567\
8901234567890123456789012345678901234567890123456789012345678901234567890123456\
7890123456789012345678901234567890123456789012345678901234567890123456789012345\
6789012345678901234567890123456789012345678901234567890123456789012345678901234\
5678901234567890";
}
?>
------------------------------------------------------------------------
[2007-01-30 14:03:32] [EMAIL PROTECTED]
Please try PHP_5_2 snapshot. It already uses litle bit different
"best-fit" implementation and this script takes reasonable time (near
the same as 5.1).
------------------------------------------------------------------------
[2007-01-29 13:24:47] thuejk at gmail dot com
I added a few printf's and found out that the $num generated "holes" in
the memory are 384 big.
Zend has a special system for small allocations, but that only works
for holes below ZEND_MM_SMALL_SIZE=280.
The $num allocations at the end, which cause the problems, and 40 or 88
long.
Note that these numbers are from a 64-bit machine, which of course has
8-byte pointers, and so a larger MM overhead. (The bug does also occur
on 32bit machines)
One temporary solution could be to raise
#define ZEND_MM_NUM_BUCKETS 32
from which ZEND_MM_SMALL_SIZE is defined, so that ZEND_MM_SMALL_SIZE is
made twice or perhaps 4 times as big. (I got a segfault when I tried
that, haven't looked into why)
A better solution would be to organize the free blocks in a balanced
tree, instead of a linear list which has to be traversed.
------------------------------------------------------------------------
[2007-01-29 01:42:28] thuejk at gmail dot com
Ok, after a good deal of work I actually tracked down the problem to
fragmentation in your memory management.
This understanding enabled me to construct a simpler test case. Note
that the problem is unrelated to database function, and this new
testcase does not require any database.
<?php
//According to my calculations, setting $num=18000000 should make this
script take ~one year.
$num = 100000;
$a = Array();
for ($i=0; $i<$num; $i++) {
$a[$i] = Array(1);
}
/* The underlying memory now looks something like
* row structure
* row value
* row structure
* ...
*/
$b = Array();
for ($i=0; $i<$num; $i++) {
$b[$i] = $a[$i][0];
}
/* The b rows are allocated
*
* Though I haven't checked it in the code, I have reason to believe
* that the values inserted into the b rows are pointers to the same
* memory allocated for the values in the a rows
*/
unset($a);
/* The a rows are unallocated, but the values are still references from
$b, so the memory map looks something like
* row value
* free_block
* row_value
* free_block
* ...
*
* repeated $num times.
*/
/* Now, for each memory allocation for a row, PHP runs through all
* free blocks checking for a best fit. Since there are $num free
* blocks, this takes time. This is done by the function
* _zend_mm_alloc_int in PHP 5.2.0 Zend/zend_alloc.c :
*
* end = &heap->free_buckets[0];
* for (p = end->next_free_block; p != end; p = p->next_free_block) {
* size_t s = ZEND_MM_FREE_BLOCK_SIZE(p);
* if (s > true_size) {
* if (s < best_size) { // better fit
* best_fit = p;
* best_size = s;
* }
* } else if (s == true_size) {
* // Found "big" free block of exactly the same size
* best_fit = p;
* goto zend_mm_finished_searching_for_block;
* }
* }
*/
$c = Array();
for ($i=0; $i<$num; $i++) {
$c[$i] = 1;
}
?>
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
http://bugs.php.net/40261
--
Edit this bug report at http://bugs.php.net/?id=40261&edit=1
