[PHP] Weird slowdown using complex functions in a loop
Hi, I'm trying to write a script to do some basic XML validation - nothing fancy, basically I just close open tags and check the arguments. The script runs fine, except that, when I use it in a loop (I need to validate roughly 200.000 posts), it gets progressively slower - like, the first ten calls to the function need around 0.01 seconds, next ten take 0.03, then it's 0.1 and so on. I've tried disabling parts of the code, I've tried profiling with dbg, I've tried manually optimizing: nothing. I managed to reduce the slowdown a bit, but it's still there, and very evident (as soon as I reach around a hundred posts, a single function call takes as long as 10 seconds). It may be interesting to note that I had to recode it from scratch: I wrote a class to do the same processing, and *that* ate memory like crazy, ending up with nearly 2 GB of used memory (according to XP taskman). If anybody has come across a similar problem, I'd love to know what did they do to solve it, but a suggestion on how to pinpoint the cause of the slowdown would be great as well. Uh, yes: I'm using PHP 4.2.2 and Apache 1.3.24 on an XP machine (but I've tried using PHP 4.3.0 and Apache 2, with the same results). pgcd -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Weird slowdown using complex functions in a loop
where are you storing the information that you are cleaning once it is cleaned? are you appending it to a variable in the script? On Wed, 2003-03-12 at 00:26, pgcd wrote: Hi, I'm trying to write a script to do some basic XML validation - nothing fancy, basically I just close open tags and check the arguments. The script runs fine, except that, when I use it in a loop (I need to validate roughly 200.000 posts), it gets progressively slower - like, the first ten calls to the function need around 0.01 seconds, next ten take 0.03, then it's 0.1 and so on. I've tried disabling parts of the code, I've tried profiling with dbg, I've tried manually optimizing: nothing. I managed to reduce the slowdown a bit, but it's still there, and very evident (as soon as I reach around a hundred posts, a single function call takes as long as 10 seconds). It may be interesting to note that I had to recode it from scratch: I wrote a class to do the same processing, and *that* ate memory like crazy, ending up with nearly 2 GB of used memory (according to XP taskman). If anybody has come across a similar problem, I'd love to know what did they do to solve it, but a suggestion on how to pinpoint the cause of the slowdown would be great as well. Uh, yes: I'm using PHP 4.2.2 and Apache 1.3.24 on an XP machine (but I've tried using PHP 4.3.0 and Apache 2, with the same results). pgcd -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Weird slowdown using complex functions in a loop
Actually, the concept is the following: - split the text into a number of elements (tags and text between tags) and put them into an array - fix each element (text and tags) to prevent attributes and entities from breaking the parser - check all the elements to make sure that they are either self-closing (text or empty tags) or matched by a closing tag somewhere - append sequentially the text from all the elements to $result - empty the elements array - return $result I honestly don't see how this can lead to a slowdown... - Original Message - From: Jimmy Brake [EMAIL PROTECTED] To: pgcd [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Wednesday, March 12, 2003 2:16 PM Subject: Re: [PHP] Weird slowdown using complex functions in a loop where are you storing the information that you are cleaning once it is cleaned? are you appending it to a variable in the script? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
