ID: 48525 Updated by: paj...@php.net Reported By: i-devolder at jm-bruneau dot be -Status: Open +Status: Assigned Bug Type: PDO related Operating System: Linux 2.6.26 ( Debian ) PHP Version: 5.3.0RC2 -Assigned To: +Assigned To: andrey New Comment:
This change has been disabled in RC4 as it introduced a BC break. However I have to say that it makes no sense to introduce it in the 1st place only to give the user the feeling that he is doing it right. As Andrey said, libmysql memory usage was not part of the results of memory_get_usage, just like any other external libraries memory usages. There is no bug per se. I would suggest to bogus this report :) Previous Comments: ------------------------------------------------------------------------ [2009-06-16 13:13:08] and...@php.net I committed a change to the 5.3 branch with the following comment: ---------- Log: Memory usage optimisation. mysqlnd is not libmysql. mysqlnd does use the Zend allocator, which means that is easier to hit memory_limit if you have big stored (buffered) result sets. Before with libmysql you won't hit memory_limit because libmysql uses libc's allocator and nothing is checked. Now, with mysqlnd the situation is stricter and it is easier to hit memory_limit. We try to optimize for big result sets. If a result set is larger than 10 rows we will start freeing some data to keep memory usage after 10 rows constant. This will help in the cases where a buffered result set is scrolled forward only and just only once, or mysqlnd will need to decode data from the network buffers again - yes, it is a trade-off between CPU time and memory size. The best for big result sets is of course using unbuffered queries - for comparison : 3 Million rows with buffered take at least 180MB, with buffered you will stay at 3MB, and unbuffered will be just 7-8% slower. ----------- According to Johannes it doesn't help PDO that much as it does help ext/mysql and ext/mysqli because of the architecture of PDO that forces mysqlnd to do a lot of extra work because of one call. And because 5.3.0 is just antes portas no change will be done to PDO before releasing 5.3.0. There is a ToDo item for PDO/PHP5.3.1 ------------------------------------------------------------------------ [2009-06-15 11:06:40] and...@php.net If you build without mysqlnd then you use libmysql and memory_get_usage() doesn't count the memory allocated by libmysql. So memory_get_usage() can give you a hint how much memory is used by the script but is not the ultimative source of informaion. I check last week mysqlnd with debug enabled and in some cases where mysqlnd wants 52 bytes to be allocated Zend allocates 102, which means the memory usage is doubled. Can't say exactly for optimized builds, could be different. ------------------------------------------------------------------------ [2009-06-11 10:10:51] i-devolder at jm-bruneau dot be the problem also occurs with mysqli My Colleague rebuilt the latest trunk without mysqlnd there we have a consistent memory usage so the problem is in the mysqlnd ------------------------------------------------------------------------ [2009-06-11 07:04:49] i-devolder at jm-bruneau dot be Description: ------------ When running a query the memoryusage keeps on rising, discovered with huge resultsets Reproduce code: --------------- $test = new PDO( $dsn , $usr , $pss ); $sql = "SELECT * FROM tech_eciffOkcab.crawlPriceComp"; $stmt = $test->prepare( $sql ); $stmt->execute(); while( $row = $stmt->fetch( PDO::FETCH_ASSOC ) ) { echo memory_get_usage()."\n"; } Expected result: ---------------- 57752 57752 57752 57752 57752 57752 57752 57752 57752 57752 output from php 5.2.9, memory usage stays the same Actual result: -------------- 33552328 33552440 33552552 33552664 33552776 33552888 33553000 PHP Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 20 bytes) in /home/idev/web/private/cmdscripts/testmem.php on line 16 output from php 5.3RC2, memory usage keeps increasing ( this is on a table with 1.2 million records ------------------------------------------------------------------------ -- Edit this bug report at http://bugs.php.net/?id=48525&edit=1