#48525 [Opn]: PDO_mysql memory leak
ID: 48525
Updated by: and...@php.net
Reported By: i-devolder at jm-bruneau dot be
Status: Open
Bug Type: PDO related
Operating System: Linux 2.6.26 ( Debian )
PHP Version: 5.3.0RC2
New Comment:
I committed a change to the 5.3 branch with the following comment:
--
Log:
Memory usage optimisation. mysqlnd is not libmysql. mysqlnd does use
the Zend allocator, which means that is easier to hit memory_limit if
you have big stored (buffered) result sets. Before with libmysql you
won't hit memory_limit because libmysql uses libc's allocator and
nothing is checked. Now, with mysqlnd the situation is stricter and it
is easier to hit memory_limit. We try to optimize for big result sets.
If a result set is larger than 10 rows we will start freeing some data
to keep memory usage after 10 rows constant. This will help in the cases
where a buffered result set is scrolled forward only and just only once,
or mysqlnd will need to decode data from the network buffers again -
yes, it is a trade-off between CPU time and memory size. The best for
big result sets is of course using unbuffered queries - for comparison :
3 Million rows with buffered take at least 180MB, with buffered you will
stay at 3MB, and unbuffered will be just 7-8% slower.
---
According to Johannes it doesn't help PDO that much as it does help
ext/mysql and ext/mysqli because of the architecture of PDO that forces
mysqlnd to do a lot of extra work because of one call. And because 5.3.0
is just antes portas no change will be done to PDO before releasing
5.3.0. There is a ToDo item for PDO/PHP5.3.1
Previous Comments:
[2009-06-15 11:06:40] and...@php.net
If you build without mysqlnd then you use libmysql and
memory_get_usage() doesn't count the memory allocated by libmysql. So
memory_get_usage() can give you a hint how much memory is used by the
script but is not the ultimative source of informaion.
I check last week mysqlnd with debug enabled and in some cases where
mysqlnd wants 52 bytes to be allocated Zend allocates 102, which means
the memory usage is doubled. Can't say exactly for optimized builds,
could be different.
[2009-06-11 10:10:51] i-devolder at jm-bruneau dot be
the problem also occurs with mysqli
My Colleague rebuilt the latest trunk without mysqlnd
there we have a consistent memory usage
so the problem is in the mysqlnd
[2009-06-11 07:04:49] i-devolder at jm-bruneau dot be
Description:
When running a query the memoryusage keeps on rising, discovered with
huge resultsets
Reproduce code:
---
$test = new PDO( $dsn , $usr , $pss );
$sql = SELECT * FROM tech_eciffOkcab.crawlPriceComp;
$stmt = $test-prepare( $sql );
$stmt-execute();
while( $row = $stmt-fetch( PDO::FETCH_ASSOC ) ) {
echo memory_get_usage().\n;
}
Expected result:
57752
57752
57752
57752
57752
57752
57752
57752
57752
57752
output from php 5.2.9, memory usage stays the same
Actual result:
--
33552328
33552440
33552552
33552664
33552776
33552888
33553000
PHP Fatal error: Allowed memory size of 33554432 bytes exhausted
(tried to allocate 20 bytes) in
/home/idev/web/private/cmdscripts/testmem.php on line 16
output from php 5.3RC2, memory usage keeps increasing ( this is on a
table with 1.2 million records
--
Edit this bug report at http://bugs.php.net/?id=48525edit=1
#48525 [Opn]: PDO_mysql memory leak
ID: 48525
Updated by: and...@php.net
Reported By: i-devolder at jm-bruneau dot be
Status: Open
Bug Type: PDO related
Operating System: Linux 2.6.26 ( Debian )
PHP Version: 5.3.0RC2
New Comment:
If you build without mysqlnd then you use libmysql and
memory_get_usage() doesn't count the memory allocated by libmysql. So
memory_get_usage() can give you a hint how much memory is used by the
script but is not the ultimative source of informaion.
I check last week mysqlnd with debug enabled and in some cases where
mysqlnd wants 52 bytes to be allocated Zend allocates 102, which means
the memory usage is doubled. Can't say exactly for optimized builds,
could be different.
Previous Comments:
[2009-06-11 10:10:51] i-devolder at jm-bruneau dot be
the problem also occurs with mysqli
My Colleague rebuilt the latest trunk without mysqlnd
there we have a consistent memory usage
so the problem is in the mysqlnd
[2009-06-11 07:04:49] i-devolder at jm-bruneau dot be
Description:
When running a query the memoryusage keeps on rising, discovered with
huge resultsets
Reproduce code:
---
$test = new PDO( $dsn , $usr , $pss );
$sql = SELECT * FROM tech_eciffOkcab.crawlPriceComp;
$stmt = $test-prepare( $sql );
$stmt-execute();
while( $row = $stmt-fetch( PDO::FETCH_ASSOC ) ) {
echo memory_get_usage().\n;
}
Expected result:
57752
57752
57752
57752
57752
57752
57752
57752
57752
57752
output from php 5.2.9, memory usage stays the same
Actual result:
--
33552328
33552440
33552552
33552664
33552776
33552888
33553000
PHP Fatal error: Allowed memory size of 33554432 bytes exhausted
(tried to allocate 20 bytes) in
/home/idev/web/private/cmdscripts/testmem.php on line 16
output from php 5.3RC2, memory usage keeps increasing ( this is on a
table with 1.2 million records
--
Edit this bug report at http://bugs.php.net/?id=48525edit=1
#48525 [Opn]: PDO_mysql memory leak
ID: 48525
User updated by: i-devolder at jm-bruneau dot be
Reported By: i-devolder at jm-bruneau dot be
Status: Open
Bug Type: PDO related
Operating System: Linux 2.6.26 ( Debian )
PHP Version: 5.3.0RC2
New Comment:
the problem also occurs with mysqli
My Colleague rebuilt the latest trunk without mysqlnd
there we have a consistent memory usage
so the problem is in the mysqlnd
Previous Comments:
[2009-06-11 07:04:49] i-devolder at jm-bruneau dot be
Description:
When running a query the memoryusage keeps on rising, discovered with
huge resultsets
Reproduce code:
---
$test = new PDO( $dsn , $usr , $pss );
$sql = SELECT * FROM tech_eciffOkcab.crawlPriceComp;
$stmt = $test-prepare( $sql );
$stmt-execute();
while( $row = $stmt-fetch( PDO::FETCH_ASSOC ) ) {
echo memory_get_usage().\n;
}
Expected result:
57752
57752
57752
57752
57752
57752
57752
57752
57752
57752
output from php 5.2.9, memory usage stays the same
Actual result:
--
33552328
33552440
33552552
33552664
33552776
33552888
33553000
PHP Fatal error: Allowed memory size of 33554432 bytes exhausted
(tried to allocate 20 bytes) in
/home/idev/web/private/cmdscripts/testmem.php on line 16
output from php 5.3RC2, memory usage keeps increasing ( this is on a
table with 1.2 million records
--
Edit this bug report at http://bugs.php.net/?id=48525edit=1
