[PHP-DEV] Bug #10299 Updated: CPU and Memory Spike
ID: 10299
User Update by: [EMAIL PROTECTED]
Status: Critical
Bug Type: Performance problem
Operating system: FreeBSD 4.2 Stable
PHP Version: 4.0.4pl1
Description: CPU and Memory Spike
What I found out is that using a lot of include_once, require_once + session will
trigger the CPU and memory spike. Add database access to the script can speed up the
frequency of the spike. And I also found out require_once DO leak memory. pstat -T
show that file in use will always increase. Somehow require_once forgot to close the
file after use. lsof can give you a clear picture when you use require_once()
I changed my code using cookie and require() statement now. And I delete all
sess_files in /tmp seems to correct the problem. This is the only change I made to
eliminate the problem.
I can't reproduce the error now because the code that have problems have been changed
and I don't have the backup. But the combination of A lot of include_once,
require_once, and session will trigger the spike, I am pretty sure because after I
changed all that, the problem disappear. Old session file in /tmp also contribute to
the problem as well.
BTW, the last time I checked, CVS version 4.06 in late April still have the problem.
Previous Comments:
---
[2001-05-08 19:26:01] [EMAIL PROTECTED]
Is this fairly reproducible?
If it is, any chance you could build Apache with debug information (--enable-debug in
configure), attach to an offending process (gdb /path/to/httpd PID), and run a
backtrace ('bt')?
---
[2001-05-07 12:44:51] [EMAIL PROTECTED]
marking critical for php 4.0.6
---
[2001-04-14 09:00:57] [EMAIL PROTECTED]
I found the problem. since the script I use have a lot of include and require
statement. just a script include a bunch of files and keep reloading the page will
kill the server in no time. I attach a sample config file I usually include, just
make a test.php and include this file like 10 times to simulate a lot of include
files. And then, just keep reloading test.php. You should see the script take a lot
of CPU time and eventually the httpd eat all the CPU time and the process hang. From
command top, one of the httpd process keep eating up CPU time.
The following is the sample config file
?
/*Root directory */
$config[root_path] = /usr2/home/business/test/www;
$config[w3b_path] = /usr2/home/business/test/www/wwwthreads;
$config[data_path] = /usr2/home/business/test/data;
$config[rdf_path] = /usr2/home/business/test/www/rdf;
/* database */
include $config[root_path]./db.php;
/* global */
$config[sitename] = www.test.net;
$config[root_url] = http://www.test.net/;;
// Cookie Setting
$config[cookie_path] = /; # web root directory
$config[CookieURL] = .test.net;
$config[cookieTTL] = 7200; // in seconds
$config[minpass]= 3; # Minimum user password length in characters
/* language and locale */
$config[language] = english;# [english] english, [big5] BIG5 code
$config[locale] = english;# Used for translation of date-related stuff
/* story control */
$config[commentmax] = 4096;
$config[abstractlen]= 4996; #length of abstract show on first page
$config[icon_width] = 100; # Topic icons width
$config[timezone] = HKT; #Time zone
$config[limitnews] = 10; # default number of stories per page
$config[older_story_num]= 10; # number of older stories to display
$config[anon_users] = 0; # 1 to allow anonymous postings
# 0 to disable
/* theme control */
$config[def_theme] = Main; # name of default theme (see themes/ directory for
list)
/* mail settings */
$config[use_fake_email] = 1; # Displays fake email addresses for users instead of the
real ones
$config[notifyEmail]= [EMAIL PROTECTED]; #address where you want to
receive notice
//$config[CensorList] =
array('fuck','cunt','fucker','fucking','pussy','cock','c0ck','cum','bitch','shit');
$config[CensorList] = array(/fuck/s,/fuck/s,/fuck/s,
/[a-z]*fuck[a-z]*/is,
/[a-z]*cunt[a-z]*/is,
/[a-z]*pussy[a-z]*/is,
/[a-z]*cock[a-z]*/is,
/[a-z]*c0ck[a-z]*/is,
/[a-z]*shit[a-z]*/is,
/[a-z]*bitch[a-z]*/is,
/[a-z]*diu[a-z]*/is
);
# See also specific LOCALE definitions
$config[CensorMode] = 3; # 0: no filtering
# 1: exact match
# 2: match word beginnings
Re: [PHP-DEV] Bug #10299 Updated: CPU and Memory Spike
I am not quite sure, but you can give a try. Use require_once() include
like 10 dummy config file, reload the script a couple times, and use lsof
to check if your config files are still open.
Derek
At 03:04 PM 5/9/2001 +1000, Cameron wrote:
i dont suppose this is a session garbage collection??
Cameron
[EMAIL PROTECTED] wrote:
ID: 10299
User Update by: [EMAIL PROTECTED]
Status: Critical
Bug Type: Performance problem
Operating system: FreeBSD 4.2 Stable
PHP Version: 4.0.4pl1
Description: CPU and Memory Spike
What I found out is that using a lot of include_once, require_once +
session will trigger the CPU and memory spike. Add database access to
the script can speed up the frequency of the spike. And I also found out
require_once DO leak memory. pstat -T show that file in use will always
increase. Somehow require_once forgot to close the file after use. lsof
can give you a clear picture when you use require_once()
I changed my code using cookie and require() statement now. And I
delete all sess_files in /tmp seems to correct the problem. This is the
only change I made to eliminate the problem.
I can't reproduce the error now because the code that have problems
have been changed and I don't have the backup. But the combination of A
lot of include_once, require_once, and session will trigger the spike, I
am pretty sure because after I changed all that, the problem
disappear. Old session file in /tmp also contribute to the problem as well.
BTW, the last time I checked, CVS version 4.06 in late April still have
the problem.
Previous Comments:
---
[2001-05-08 19:26:01] [EMAIL PROTECTED]
Is this fairly reproducible?
If it is, any chance you could build Apache with debug information
(--enable-debug in configure), attach to an offending process (gdb
/path/to/httpd PID), and run a backtrace ('bt')?
---
[2001-05-07 12:44:51] [EMAIL PROTECTED]
marking critical for php 4.0.6
---
[2001-04-14 09:00:57] [EMAIL PROTECTED]
I found the problem. since the script I use have a lot of include and
require statement. just a script include a bunch of files and keep
reloading the page will kill the server in no time. I attach a sample
config file I usually include, just make a test.php and include this file
like 10 times to simulate a lot of include files. And then, just keep
reloading test.php. You should see the script take a lot of CPU time and
eventually the httpd eat all the CPU time and the process hang. From
command top, one of the httpd process keep eating up CPU time.
The following is the sample config file
?
/*Root directory */
$config[root_path] = /usr2/home/business/test/www;
$config[w3b_path] = /usr2/home/business/test/www/wwwthreads;
$config[data_path] = /usr2/home/business/test/data;
$config[rdf_path] = /usr2/home/business/test/www/rdf;
/* database */
include $config[root_path]./db.php;
/* global */
$config[sitename] = www.test.net;
$config[root_url] = http://www.test.net/;;
// Cookie Setting
$config[cookie_path] = /; # web root directory
$config[CookieURL] = .test.net;
$config[cookieTTL] = 7200; // in seconds
$config[minpass]= 3; # Minimum user password length in characters
/* language and locale */
$config[language] = english;# [english] english, [big5]
BIG5 code
$config[locale] = english;# Used for translation of
date-related stuff
/* story control */
$config[commentmax] = 4096;
$config[abstractlen]= 4996; #length of abstract show on first page
$config[icon_width] = 100; # Topic icons width
$config[timezone] = HKT; #Time zone
$config[limitnews] = 10; # default number of stories per page
$config[older_story_num]= 10; # number of older stories to
display
$config[anon_users] = 0; # 1 to allow anonymous postings
# 0 to disable
/* theme control */
$config[def_theme] = Main; # name of default theme (see themes/
directory for list)
/* mail settings */
$config[use_fake_email] = 1; # Displays fake email addresses for users
instead of the real ones
$config[notifyEmail]= [EMAIL PROTECTED]; #address where
you want to receive notice
//$config[CensorList] =
array('fuck','cunt','fucker','fucking','pussy','cock','c0ck','cum','bitch','shit');
$config[CensorList] = array(/fuck/s,/fuck/s,/fuck/s,
/[a-z]*fuck[a-z]*/is,
/[a-z]*cunt[a-z]*/is,
/[a-z]*pussy[a-z]*/is,
/[a-z]*cock[a-z]*/is,
Re: [PHP-DEV] RE: PHP 4.0 Bug #10299 Updated: CPU and Memory Spike
That's my whole point. PHP should be able handle it pretty well. But Chris Adams said my setup can only support 10 concurrent connections. That's why I am curious what is wrong here. I am pretty sure PHP can handle the load, but somehow something trigger the CPU spike. I just can't find out why. rgd, Derek -Original Message- From: Derick Rethans To: Derek Leung Cc: '[EMAIL PROTECTED]' Sent: 4/15/01 9:34 AM Subject: Re: [PHP-DEV] RE: PHP 4.0 Bug #10299 Updated: CPU and Memory Spike On Sun, 15 Apr 2001, Derek Leung wrote: I did try the snap shot 4/14/2001 4.06dev, it didnt' solve the problem. If 256mb ram , dual 400mhz can only serve around 10 concurrent clients, this is a MAJOR issue for PHP. I guess any web language like PERL, Cold fusion, JSP, servlet can do a LOT more than this with the equipment I am using right now. Well, my Pentium 133 with 48MB's of RAM even gets around 25 concurrent connections... Derick Rethans -- PHP Development Mailing List http://www.php.net/ To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] To contact the list administrators, e-mail: [EMAIL PROTECTED]
[PHP-DEV] PHP 4.0 Bug #10299 Updated: CPU and Memory Spike
ID: 10299 User Update by: [EMAIL PROTECTED] Status: Open Bug Type: Performance problem Description: CPU and Memory Spike This is vmstat show if I keep reloading the script procs memory pagedisks faults cpu r b w avm fre flt re pi po fr sr da0 da1 in sy cs us sy id 1 4 0 73860 134324 0 0 0 0 0 2 0 382 3804 258 16 1 83 0 4 0 73860 134244 0 0 0 0 0 0 0 357 3789 245 15 4 81 1 4 0 73860 134244 0 0 0 0 0 0 0 373 3763 263 14 3 83 2 4 0 73860 13416 142 0 0 0 139 0 0 0 402 6155 285 26 4 70 0 4 0 73860 134084 0 0 0 4 0 11 0 366 3179 220 11 2 87 1 4 0 73860 134004 0 0 0 0 0 0 0 358 3943 257 15 3 82 2 4 0 73860 134004 0 0 0 0 0 0 0 340 4201 240 12 42 45 2 4 0 73860 134004 0 0 0 0 0 0 0 240 1842 90 0 100 0 2 4 0 73116 134004 0 0 0 0 0 19 0 265 1802 98 0 100 0 2 4 0 73116 134004 0 0 0 0 0 0 0 241 1753 93 0 100 0 2 4 0 73116 134004 0 0 0 0 0 0 0 239 1729 96 0 100 0 2 4 0 73116 134004 0 0 0 0 0 0 0 241 1612 94 0 100 0 2 4 0 73116 134004 0 0 0 0 0 0 0 243 1576 97 0 100 0 2 4 0 73116 134004 0 0 0 0 0 0 0 243 1541 98 0 100 0 2 4 0 73116 134004 0 0 0 0 0 0 0 239 1487 102 0 100 0 2 4 0 73116 134004 0 0 0 0 0 0 0 241 1401 93 0 100 0 2 4 0 73116 134004 0 0 0 0 0 0 0 251 1405 100 0 100 0 2 4 0 73116 134004 0 0 0 0 0 0 0 246 1338 101 0 100 0 2 4 0 73116 134004 0 0 0 0 0 0 0 238 1297 94 0 100 0 2 4 0 73860 134004 0 0 0 0 0 0 2 243 1282 100 0 100 0 2 4 0 73860 134004 0 0 0 0 0 0 0 239 1223 89 0 100 0 2 4 0 73860 134004 0 0 0 2 0 9 16 300 1180 92 0 100 0 procs memory pagedisks faults cpu r b w avm fre flt re pi po fr sr da0 da1 in sy cs us sy id 3 4 0 73860 134004 0 0 0 0 0 0 0 251 1156 90 0 100 0 3 4 0 50296 134004 0 0 0 0 0 6 0 258 1116 93 0 100 0 2 4 0 50296 134004 0 0 0 0 0 0 0 240 1108 83 0 100 0 3 4 0 50296 134004 0 0 0 0 0 0 0 245 1064 92 0 100 0 3 4 0 46988 134004 0 0 0 0 0 0 0 242 1031 83 0 100 0 3 4 0 46988 134004 0 0 0 0 0 0 0 242 989 82 0 100 0 2 4 0 46988 134004 0 0 0 0 0 0 0 243 1010 85 0 100 0 2 4 0 46988 134004 0 0 0 0 0 1 0 241 985 81 0 100 0 2 4 0 46988 134004 0 0 0 0 0 0 0 241 971 89 1 99 0 2 4 0 46988 134004 0 0 0 0 0 0 0 243 940 74 0 100 0 3 4 0 46988 134004 0 0 0 2 0 8 0 250 926 80 0 100 0 2 4 0 46988 134004 0 0 0 0 0 1 0 245 930 88 0 100 0 3 4 0 46988 134004 0 0 0 0 0 0 0 240 905 80 0 100 0 2 4 0 46988 134004 0 0 0 0 0 7 0 253 920 100 0 98 2 3 4 0 47332 13056 90 0 0 0 0 0 0 0 239 2113 116 3 97 0 2 4 0 46588 130484 0 0 0 0 0 1 0 243 2334 103 1 99 0 1 4 0 46244 133924 0 0 0 86 0 8 0 276 4383 140 7 19 74 1 4 0 47896 13184 59 0 0 0 0 0 2 0 275 621 56 7 1 92 0 4 0 47896 13152 20 0 0 0 0 0 1 0 253 471 46 6 1 93 1 4 0 47492 13372 18 0 0 0 64 0 0 0 251 478 43 0 1 99 0 4 0 47492 133724 0 0 0 0 0 0 0 235 440 40 0 0 100 0 4 0 47492 133724 0 0 0 0 0 0 0 238 442 40 0 2 98 The CPU time drop from 80 all the way to zero and make the Apache hang for like 30 seconds and the resume back again. It happens VERY often if I only have like 30 users surf the website at the same time. It pretty much bring the web server down if I have around 50 users surft the site Previous Comments: --- [2001-04-11 18:03:59] [EMAIL PROTECTED] I am running on FreeBSD 4.2 Stable, Apache 1.3.19 + PHP 4.0.4pl1 + Modssl on a Dual P2 400mhz with 256mb ram. From time to time, one of the httpd process will eat up all the CPU time and memory, and after around 5 to 10 seconds, it goes back to normal Even a Very simple script like ? echo "test"; ? can make it happen. 1 out of 25 reload of the above script can make the spike happen. But the web server do not crash though. If I run on a script with heavy database access, it happen 1 out of 10 if I press reload button continously. BTW, is that normal for Apache to take up 30% of the CPU time to excute
[PHP-DEV] PHP 4.0 Bug #10299 Updated: CPU and Memory Spike
ID: 10299
User Update by: [EMAIL PROTECTED]
Status: Open
Bug Type: Performance problem
Description: CPU and Memory Spike
I found the problem. since the script I use have a lot of include and require
statement. just a script include a bunch of files and keep reloading the page will
kill the server in no time. I attach a sample config file I usually include, just
make a test.php and include this file like 10 times to simulate a lot of include
files. And then, just keep reloading test.php. You should see the script take a lot
of CPU time and eventually the httpd eat all the CPU time and the process hang. From
command "top", one of the httpd process keep eating up CPU time.
The following is the sample config file
?
/*Root directory */
$config[root_path] = "/usr2/home/business/test/www";
$config[w3b_path] = "/usr2/home/business/test/www/wwwthreads";
$config[data_path] = "/usr2/home/business/test/data";
$config[rdf_path] = "/usr2/home/business/test/www/rdf";
/* database */
include $config[root_path]."/db.php";
/* global */
$config[sitename] = "www.test.net";
$config[root_url] = "http://www.test.net/";
// Cookie Setting
$config[cookie_path] = "/"; # web root directory
$config[CookieURL] = ".test.net";
$config[cookieTTL] = 7200; // in seconds
$config[minpass]= 3; # Minimum user password length in characters
/* language and locale */
$config[language] = "english";# [english] english, [big5] BIG5 code
$config[locale] = "english";# Used for translation of date-related stuff
/* story control */
$config[commentmax] = 4096;
$config[abstractlen]= 4996; #length of abstract show on first page
$config[icon_width] = 100; # Topic icons width
$config[timezone] = "HKT"; #Time zone
$config[limitnews] = 10; # default number of stories per page
$config[older_story_num]= 10; # number of older stories to display
$config[anon_users] = 0; # 1 to allow anonymous postings
# 0 to disable
/* theme control */
$config[def_theme] = "Main"; # name of default theme (see themes/ directory for
list)
/* mail settings */
$config[use_fake_email] = 1; # Displays fake email addresses for users instead of the
real ones
$config[notifyEmail]= "[EMAIL PROTECTED]"; #address where you want to
receive notice
//$config[CensorList] =
array('fuck','cunt','fucker','fucking','pussy','cock','c0ck','cum','bitch','shit');
$config[CensorList] = array("/fuck/s","/fuck/s","/fuck/s",
"/[a-z]*fuck[a-z]*/is",
"/[a-z]*cunt[a-z]*/is",
"/[a-z]*pussy[a-z]*/is",
"/[a-z]*cock[a-z]*/is",
"/[a-z]*c0ck[a-z]*/is",
"/[a-z]*shit[a-z]*/is",
"/[a-z]*bitch[a-z]*/is",
"/[a-z]*diu[a-z]*/is"
);
# See also specific LOCALE definitions
$config[CensorMode] = 3; # 0: no filtering
# 1: exact match
# 2: match word beginnings
# 3: match string anywhere in words
$config[CensorReplace] = "[CENSORED]" ;
/* allowable HTML tags. Used by check_html()
Happily lifted (with mods) from phpslash */
$config[AllowableHTML] = array("p"=1, # 2 means accept all qualifiers: foo bar
"b"=1,# 1 means accept the tag only: foo
"i"=1,
"a"=2,
"em"=1,
"br"=1,
"strong"=1,
"blockquote"=1,
"img"=2,
"tt"=1,
"hr"=1,
"li"=1,
"ol"=1,
"div"=1,
"ul"=1);
$config[AllowableHTMLtag] =
"b,i,a,em,br,strong,blockquote,tt,hr,li,ol,div,ul";
/* poll settings */
$config[maxOptions] = 12; # maximal number of options allowed
$config[BarScale] = 1;# scale of result bar (in multiples of 100 pixels)
$config[warnCheaters] = 1;# 1 - a notice issued on vote second try
# 0 - silently ignores the vote
$config[maxOptions] = 10;
/* story stuff */
$reasons = array('As is', # "Normal"
'Offtopic', # Bad Responses
'Flamebait',
'Troll',
'Redundant',
'Insightful', # Good Responses
'Interesting',
'Informative',
'Funny',
'Overrated',# Last two are "special"
'Underrated');
if ($config[dateString] ==
[PHP-DEV] Bug #10299: CPU and Memory Spike
From: [EMAIL PROTECTED] Operating system: FreeBSD 4.2 Stable PHP version: 4.0.4pl1 PHP Bug Type: Performance problem Bug description: CPU and Memory Spike I am running on FreeBSD 4.2 Stable, Apache 1.3.19 + PHP 4.0.4pl1 + Modssl on a Dual P2 400mhz with 256mb ram. From time to time, one of the httpd process will eat up all the CPU time and memory, and after around 5 to 10 seconds, it goes back to normal Even a Very simple script like ? echo "test"; ? can make it happen. 1 out of 25 reload of the above script can make the spike happen. But the web server do not crash though. If I run on a script with heavy database access, it happen 1 out of 10 if I press reload button continously. BTW, is that normal for Apache to take up 30% of the CPU time to excute one single PHP-database-driven page? I search the bugs database and I found another user have the exact problem like what I am having right now. His bug id is #9154. This is how I configure PHP ./configure --with-mysql=/usr/local/mysql --with-apxs=/usr/local/apache/bin/apxs --with-config-file-path=/usr/local/etc --enable-track-vars --enable-ftp --with-gd=/usr/local --with-pcre-regex=/usr/local/ --with-jpeg-dir=/usr/local --with-xpm-dir=/usr/X11R6 With or Without modssl enable does not make a difference. Both can cause the spike. -- Edit Bug report at: http://bugs.php.net/?id=10299edit=1 -- PHP Development Mailing List http://www.php.net/ To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] To contact the list administrators, e-mail: [EMAIL PROTECTED]
