Re: [PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performance of Apache 2.0 Filter
At 09:12 PM 6/7/2002 -0700, Brian Pane wrote: In case it's helpful to the PHP developers, here are some more performance problems that I found by running a quick system call profile of PHP-4.2.1 within Apache-2.0.37-dev: * php_request_shutdown() calls shutdown_memory_manager(), which does a large number of calls to free() per request. If there's any way to get the PHP allocator to use an APR pool, that should help speed things up a lot. (The mallocs and frees are going to be especially problematic within multithreaded MPMs.) We're already doing this for Win32. Check out ZEND_DO_MALLOC/ZEND_DO_FREE/ZEND_DO_REALLOC in zend_alloc.c. Note that in Win32 we only skip the free's if we're in release mode. If we're in debug mode we use a per-thread pool but we do the frees because it's our memory leak detector. * The lex_scan() function consumed a lot of CPU time, even though in my test case (a file with no actual embedded scripting code) only the loop right after the yy_match label ever got executed. In general I've profiled PHP and the scanning stage doesn't take too much CPU time. You shouldn't test an empty html page but rather a real-life PHP script. I've profiled PHP a lot and be aware that the profiling results change a lot according to what script you're running. As it's gone through various optimizations already there isn't an easy to find bottleneck. Andi -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: [Zend Engine 2] RE: [PHP-DEV] oo != php
At 05:02 AM 6/8/2002 +0100, Michael Dransfield wrote: At 02:40 08/06/2002 +0100, you wrote: There are two reasons we repeat the 'PHP is not Java mantra': (a) Many of those requesting these changes actually DO want to see PHP as a Java with PHPish syntax. Anyone wanting PHP to be a simple or more flexable Java is barking up the wrong tree... in fact all of the people I know who Im lobbying better OO functionality in PHP for know Java and know PHP - and use both where each is best. What we are requesting is that PHP expands its OO capabilities - not change the way it does things, not do anything outlandish or stolen from another language. Think about it - all it would be is like adding an extra gear to a car. Wouldnt change the concept, the design or the idea of the car... not the make nor model... it would however give it added depth and use. I agree completely, for what its worth ;) I have a question about this whole debate. If PHP's core goal is to provide easy quick development for 'dynamic pages' , then why is there no built-in template engine? Having this would solve many of the niggling problems that are associated with current PHP-written (OO) template engines. Is it that there is no desire for such an extension? Please read the archives on this issue because I wouldn't want yet another long thread to start on php-dev. Andi -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performance of Apache 2.0 Filter
At 10:41 AM 6/8/2002 +0300, Andi Gutmans wrote: At 09:12 PM 6/7/2002 -0700, Brian Pane wrote: In case it's helpful to the PHP developers, here are some more performance problems that I found by running a quick system call profile of PHP-4.2.1 within Apache-2.0.37-dev: * php_request_shutdown() calls shutdown_memory_manager(), which does a large number of calls to free() per request. If there's any way to get the PHP allocator to use an APR pool, that should help speed things up a lot. (The mallocs and frees are going to be especially problematic within multithreaded MPMs.) We're already doing this for Win32. Check out ZEND_DO_MALLOC/ZEND_DO_FREE/ZEND_DO_REALLOC in zend_alloc.c. Note that in Win32 we only skip the free's if we're in release mode. If we're in debug mode we use a per-thread pool but we do the frees because it's our memory leak detector. I just checked and it seems like Apache APR memory pools use mutex locking. It'd be better to use functions like the Win32 ones which don't use mutex locking (as we made sure that only one thread allocates from its pool). This could be achieved by compiling apr_pools.c without APR_HAS_THREADS but I bet the default Apache 2 build has this enabled. Any ideas? Andi -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] filesystem security questions
I have a few questions about correctly implementing safe_mode, open_basedir and allow_url checks within an extension. Probably an easy question, but I have seen it implemented in various ways in different extensions and want to make sure i implement this correctly. allow_url checks: is there a standard function which produces an error if not allowed, or do I just handle this within the extension after testing with PG(allow_url_fopen)? safe_mode: this is done by testing both php_check_safe_mode_include_dir and php_checkuid if safe_mode is enabled, correct? open_basedir: this is just done with php_check_open_basedir? Is there any precedence of the safe_mode and open_basedir checks or does it need to pass both checks? Are there any other checks i am missing to implement filesystem security within an extension? Thanks, Rob -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] filesystem security questions
Hi,
On Sat, Jun 08, 2002 at 06:22:29AM -0400, Rob Richards wrote :
I have a few questions about correctly implementing safe_mode, open_basedir
and allow_url checks within an extension.
Probably an easy question, but I have seen it implemented in various ways in
different extensions and want to make sure i implement this correctly.
allow_url checks:
is there a standard function which produces an error if not allowed, or
do I just handle this within the extension after testing with
PG(allow_url_fopen)?
In CVS HEAD we've a new, generic implemention called
'streams'. The streams implementation takes care of this. See
main/streams.c in locate_url_wrapper (which is called when
you create a new stream):
if (wrapper wrapper-is_url !PG(allow_url_fopen)) {
zend_error(E_WARNING, URL file-access is disabled in the server
configuration);
return NULL;
}
Practically, there's no documentation about streams except
some mail Wez sent to php-dev a few weeks (months?) ago (and
the sources, of course), I hope you can find it in the
archives.
safe_mode:
this is done by testing both php_check_safe_mode_include_dir and
php_checkuid if safe_mode is enabled, correct?
open_basedir:
this is just done with php_check_open_basedir?
I leave this open as I'm not an expert in this field. Current
implementation seem to call php_checkuid first and then
php_check_open_basedir (grep for the functions in
ext/standard/* ).
- Markus
--
GnuPG Key: http://guru.josefine.at/~mfischer/C2272BD0.asc
--
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] filesystem security questions
At 12:42 PM 6/8/2002 +0200, Markus Fischer wrote: Practically, there's no documentation about streams except some mail Wez sent to php-dev a few weeks (months?) ago (and the sources, of course), I hope you can find it in the archives. Maybe we can get Wez to write complete documentation for the API docs? :) Andi -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] filesystem security questions
I think using stream functions may be overkill for what I need to do.
While fixing an issue with using relative paths in domxml and decided to add
filesecurity features in at the same time.
The file path comes in as a string and is sent directly to libxml as such.
also, going through the locate_url_wrapper function, it looks like it
defines acceptable protocols to use.
Also, libxml has its own routines for remote file access, so in the event a
new protocol is used (right now it is just http and ftp), should the
extension disallow the use of other protocols if it is not supported in php?
may not be an issue.
right now I am using the following to test for a remote path:
if (!strncasecmp(file,http://;, 7) || !strncasecmp(file,ftp://;, 6)) {
which could be expanded upon if additional protocols are added to libxml.
I could be wrong (wouldn't be the first time), but I am not sure if I really
need to use the streams functions as I wont be doing anything with the
returned wrapper.
Thanks
Rob
- Original Message -
From: Markus Fischer [EMAIL PROTECTED]
In CVS HEAD we've a new, generic implemention called
'streams'. The streams implementation takes care of this. See
main/streams.c in locate_url_wrapper (which is called when
you create a new stream):
if (wrapper wrapper-is_url !PG(allow_url_fopen)) {
zend_error(E_WARNING, URL file-access is disabled in the server
configuration);
return NULL;
}
--
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] filesystem security questions
Hi Rob,
At 13:22 8-6-2002, Rob Richards shared with all of us:
right now I am using the following to test for a remote path:
if (!strncasecmp(file,http://;, 7) || !strncasecmp(file,ftp://;, 6)) {
which could be expanded upon if additional protocols are added to libxml.
From a user's point of view:
Sablotron uses scheme handlers, which can be registered for a certain
protocol. This allows custom protocols to be added in php.
Would love to see the same thing apply to domxml and if eventually
resources between the different extensions could be passed arround, this
would certainly help.
One could do it like this for instance:
!ENTITY % dynamic SYSTEM
domxml://resourcename/rootnode/entityblock/entityname
%dynamic;
and vice-versa for xml-stylesheet PI's.
The remote check would be static:
if(!strncasecmp(file, file://, 7)) || !strncasecmp(file, sablot://, 9)) {
Then through a case statement of 'known' schemes and any that require
safe_mode could go
through the streams.
Any custom schemas that are registered in php, would have to call
fopen/sockets/whatever and are handled by streams/safe_mode automatically.
2c.
Met vriendelijke groeten / With kind regards,
IDG.nl
Melvyn Sopacua
Webmaster
--
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performance of Apache2.0 Filter
On Fri, 7 Jun 2002, Cliff Woolley wrote: On Fri, 7 Jun 2002, Brian Pane wrote: IMHO, that's a design flaw. Regardless of whether PHP is doing buffering, it shouldn't break up blocks of static content into small pieces--especially not as small as 400 bytes. While it's certainly valid for PHP to insert a flush bucket right before a block of embedded code (in case that code takes a long time to run), breaking static text into 400-byte chunks will usually mean that it takes *longer* for the content to reach the client, which probably defeats PHP's motivation for doing the nonbuffered output. There's code downstream, in the httpd's core_output_filter and the OS's TCP driver, that can make much better decisions about when to buffer and when not to buffer. FWIW, I totally agree here. One of the biggest problems with the way PHP handles buckets (as I'm sure has been discussed before I know) is that static content cannot remain in its native form as it goes through PHP, or at least not in very big chunks. Take as a counterexample the way mod_include deals with FILE buckets. It reads the FILE bucket (which causes the file the be MMAPed if allowed), and from there it just scans through the mmaped region, and if it finds nothing, it hands it on to the next filter still in the single-MMAP-bucket form. PHP/Zend, on the other hand, takes the file descriptor out of the file bucket, runs it through a lexical analyzer which tokenizes it up to 400 bytes at a time, runs it through the yacc-generated grammar as necessary, and handles it from there. Far more optimal would be to take the input, do a search through it for a starting tag just as mod_include does, and if none is found (or up until one is found), just tell the SAPI module to go ahead and send up to THIS point out to the client unmodified. So basically the difference between this and what we have now is that the lexer should not take each 400 byte buffer and say here is (up to) 400 bytes of inline HTML, send it to the client as-is; instead, it should be able to do something along the lines of taking the input 400 bytes at a time, and as soon as it starts seeing inline HTML, keep track of the starting offset (in bytes), and keep scanning through those 400 byte buffers in a tight loop until it finds something that's NOT inline HTML, and set the ending offset. Then it can notify PHP in one call send bytes 375-10136 to the client as-is, it's inline html. I don't believe that this is what's slowing PHP down on Apache 2. Don't forget that after the first piece of PHP code we can't optimize inline HTML any more because they can be conditional. I think that the non-mutexed per-thread memory pool would improve performance much more significantly. Andi -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] PHP 4 Bug Summary Report
PHP 4 Bug Database summary - http://bugs.php.net Num Status Summary (1564 total including feature requests) ===[*Configuration Issues] 13561 Suspended --without-pear prevent install of php-config,phpize,... 15182 Open php.ini seems to have no affect 15264 Feedback unable to include self when loaded as module 15453 Feedback PhP fails to find mcrypt API 16184 Feedback Crashes during ./configure script 16311 Feedback php.ini-dist and php.ini-recommended missing extensions 16758 Assigned Configure Libraries have been installed in and where they really are 16920 Analyzed File permissions security problem 17320 Feedback invalid php_config.h for solaris 2.7 platform ===[*Directory/Filesystem functions] 12004 Assigned problem with fopen over ftp and a related fgets 12783 Open xmlDocFile needs absolute path 14076 Open fopen() and touch() fail to create file under safe mode 14100 Open Unicode Filenames 14657 Critical Sometimes mkdir(test1, 0777); $d=dir(test1); $d-close(); rmdir(test1); 16630 Open getlastmod() returns file access time, not modification time 17229 Feedback Permissions set incorrectly on newly created directories. 17471 Open Problem with Novell Netware map drive ===[*Extensibility Functions]= 15125 Feedback pnctl_signal does not handle class's as callbacks - patch included ===[*General Issues]== 15926 Feedback sum error 16094 Feedback Unable to view .php file in the browser 17287 Open safe mode is not full safe. 17353 Open Finish glob() support for all OS (hint: flags and portability) 17543 Feedback non-printable characters are converted to spaces when printing 17648 Open syslog inserts random characters into log files ===[*Graphics related] 15662 Open iptcembed FotoWare ===[*Languages/Translation]=== 11975 Assigned mix of hebrew english ===[*Network Functions]=== 15639 Open detecting end of UDP packets 15988 Open Pending established tcp connections 16735 Open checkdnsrr claims not to be implemented ===[*PDF functions]=== 16911 Feedback Activate extention and PHP hangs ===[*Programming Data Structures]= 16304 Feedback Missing Error Messages ===[*Regular Expressions]= 17570 Open Memory leak ===[Apache related]=== 8866 Feedback Frozen applications with apache module running 9280 Open HTTP/1.1 Expect: header not honoured 9422 Open Apache hangs when Win goes to standby 10060 Open startup error msg format 11716 Open Segmentation fault(coredump) in Apache(DSO-enabled) startup 12992 Open php ignores Accept: text/vnd.wap.wml 13035 Open 404 on a .php results in SERVER ERROR when using php as a cgi 13420 Open open_basedir breaks Apache SSI xbithack 13421 Open Problematic MIME-Type application/x-httpd-php 13925 Analyzed SCRIPT_NAME in Apache 1.3.22 wrong (or at least different) 13956 Open improve robustness against bad signal(2) et al. calls from third party libs 13961 Assigned some characters in incomonig variable names are silently changed 13964 Open core dump on AIX 4.3.2 13982 Open set_time_limit (ref #13711) and ignore_user_abort affects later requests 14222 Open PHP Crashes with weird output often 14229 Open function virtual 14245 Open make install fails on apxs 14265 Open Make does not create php4lib.so. Apxs fails to copy it to apache dir 14370 Open PHP_AUTH_PW being improperly set 14401 Open Wrong include_path from Apache Directory config 14409 Open request for nonexistent file does not return 404 error 14534 Open Variables $PHP_AUTH_* is set, when use a traditional external auth mechanism 14662 Open Apache segvs on SIGHUP 14689 Open apache fails to load php due to mmap failure 14865 Feedback php4apache.dll - phpinfo error - winXP - Apache 14956 Open --with-imap-ssl hand apache start with libssl.so 15038 Open replace parameter of Header() function doesn't work in Apache module 15275 Open Apache 1.3.20 won't start with newly compiled PHP 4.1.1 15529 Open ap_cleanup_for_exec not used when creating 15579 Open Segmenation fault 15588 Open Segmentaion fault 15747 Open Undefined symbol _mail_string 15935 Open phpinfo routine mixing SCRIPT_URI and SCRIPT_URL 15954 Open problem on shutdown/oleaut32.dll 15968 Open make error when making
[PHP-DEV] On vacation
I'm on vacation until June 16th and will not read any mail until I return. Your mail concerning Assigned PHP 4 bugs reminder will be read when I'm back. Stig -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] filesystem security questions
Are you talking about your own extension or ext/domxml ? I
think (but could be wrong) that sooner or later streams
should be used everyone for consistency.
- Markus
On Sat, Jun 08, 2002 at 07:22:25AM -0400, Rob Richards wrote :
I think using stream functions may be overkill for what I need to do.
While fixing an issue with using relative paths in domxml and decided to add
filesecurity features in at the same time.
The file path comes in as a string and is sent directly to libxml as such.
also, going through the locate_url_wrapper function, it looks like it
defines acceptable protocols to use.
Also, libxml has its own routines for remote file access, so in the event a
new protocol is used (right now it is just http and ftp), should the
extension disallow the use of other protocols if it is not supported in php?
may not be an issue.
right now I am using the following to test for a remote path:
if (!strncasecmp(file,http://;, 7) || !strncasecmp(file,ftp://;, 6)) {
which could be expanded upon if additional protocols are added to libxml.
I could be wrong (wouldn't be the first time), but I am not sure if I really
need to use the streams functions as I wont be doing anything with the
returned wrapper.
Thanks
Rob
- Original Message -
From: Markus Fischer [EMAIL PROTECTED]
In CVS HEAD we've a new, generic implemention called
'streams'. The streams implementation takes care of this. See
main/streams.c in locate_url_wrapper (which is called when
you create a new stream):
if (wrapper wrapper-is_url !PG(allow_url_fopen)) {
zend_error(E_WARNING, URL file-access is disabled in the server
configuration);
return NULL;
}
--
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php
--
GnuPG Key: http://guru.josefine.at/~mfischer/C2272BD0.asc
Did I help you?http://guru.josefine.at/wish_en
Konnte ich helfen? http://guru.josefine.at/wish_de
--
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] filesystem security questions
I am talking about the domxml. After thinking about a bit and reading the post from Melvyn, I think that streams might be the way to go, but not quite positive yet. In a long term sense, I am just not sure how far domxml is going to go; meaning the discussion of the unified xml implementation. I dont know where this stands, though don't see it being rolled out in the immediate future with all that needs to be done, and assume that it will replace domxml. It looks like there will be two tracks here. One maintaining domxml and they other working on the unified implementation. With that in mind, where should the focus of development on domxml be? I would like to see it get out of experimental stage and considered stable. It may not include all the bells and whistles, but at least for the functionality it does support, it is considered solid. Rob - Original Message - From: Markus Fischer [EMAIL PROTECTED] To: Rob Richards [EMAIL PROTECTED] Cc: Php-Dev List [EMAIL PROTECTED] Sent: Saturday, June 08, 2002 10:16 AM Subject: Re: [PHP-DEV] filesystem security questions Are you talking about your own extension or ext/domxml ? I think (but could be wrong) that sooner or later streams should be used everyone for consistency. - Markus -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] CVS Account Request: thku
like to check my improvements of the OOHForms in - talked to ulf wendel and the rest oh the german php user group at the linux tag -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Zend Engine expert wanted!
At 2:57 AM +0300 6/8/02, Zeev Suraski wrote: We're on the job. I generally think you're right, we have to do some more thinking but chances are we will change the shutdown order to be reversed. Just a FYI, may want to do the same for zend_extension (reverse order). I think that is a zend_llist instead of a zend_hash, but I believe it unloads in the same order they are loaded instead of reverse. Sorry for not ack'ing earlier. No problem, just glad I confirmed the fix. Thanks for help! Cheers, Brian -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performance of Apache2.0 Filter
Looking at some more syscall call traces, I'm seeing that
the flush buckets used by php_apache_sapi_ub_write() are
causing very small packets even for script-generated output,
not just for blocks of static text.
For example, I just compared the write patterns for 2.0.37
and 1.3.24, both with the default configuration for PHP-4.2.1,
while delivering a file consisting solely of:
?phpinfo() ?
1.3:
writev(3, [{HTTP/1.1 200 OK\r\nDate: Sat, 08 J..., 4089}, {28\r\n,
4}, {tr valign=\baseline\ bgcolor=\#..., 40}, {\r\n, 2}], 4) = 4135
write(3, ffb\r\ntd bgcolor=\#ff\bas..., 4098) = 4098
writev(3, [{fdd\r\ntd bgcolor=\#ff\bhig..., 4068}, {22\r\n,
4}, {/bbr //tdtd align=\center..., 34}, {\r\n, 2}], 4) = 4108
writev(3, [{ff2\r\n/usr/sbin/sendmail -t -i /..., 4089}, {22\r\n,
4}, {/bbr //tdtd align=\center..., 34}, {\r\n, 2}], 4) = 4129
(followed by a few more 4KB writes)
2.0:
writev(8, [{HTTP/1.1 200 OK\r\nDate: Sat, 08 J..., 220}, {47\r\n,
4}, {!DOCTYPE HTML PUBLIC \-//W3C//D..., 71}, {\r\n, 2}], 4) = 297
writev(8, [{6\r\n, 3}, {head, 6}, {\r\n, 2}], 3) = 11
writev(8, [{1c\r\n, 4}, {style type=\text/css\!--\n, 28},
{\r\n, 2}], 3) = 34
writev(8, [{1d\r\n, 4}, {a { text-decoration: none; }\n, 29},
{\r\n, 2}], 3) = 35
writev(8, [{28\r\n, 4}, {a:hover { text-decoration: under..., 40},
{\r\n, 2}], 3) = 46
writev(8, [{55\r\n, 4}, {h1 { font-family: arial, helveti..., 85},
{\r\n, 2}], 3) = 91
writev(8, [{55\r\n, 4}, {h2 { font-family: arial, helveti..., 85},
{\r\n, 2}], 3) = 91
writev(8, [{49\r\n, 4}, {body, td { font-family: arial, h..., 73},
{\r\n, 2}], 3) = 79
writev(8, [{56\r\n, 4}, {th { font-family: arial, helveti..., 86},
{\r\n, 2}], 3) = 92
writev(8, [{e\r\n, 3}, {//--/style\n, 14}, {\r\n, 2}], 3) = 19
(followed by a thousand more tiny writes)
It looks like unbuffered PHP output in 1.3 is benefitting from the
httpd's own buffering. And unbuffered output in 2.0, in contrast,
is really, really not buffered, because of the flush buckets inserted
by php_apache_sapi_ub_write. And the tiny packets that result in the
2.0 case are likely a major cause of reports of slowness under 2.0.
--Brian
--
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performanceof Apache 2.0 Filter
Andi Gutmans wrote: I just checked and it seems like Apache APR memory pools use mutex locking. It'd be better to use functions like the Win32 ones which don't use mutex locking (as we made sure that only one thread allocates from its pool). This could be achieved by compiling apr_pools.c without APR_HAS_THREADS but I bet the default Apache 2 build has this enabled. Any ideas? Andi It's not as bad as it looks. :-) The APR pools do locking only in a few situations: * Creating a new pool * Creating a subpool of a pool * Destroying a pool * Adding space to an existing pool, if it's used up its initial 8KB allocation So the simplest case, the lifetime of a pool looks like: - create the pool, requiring a lock/unlock - do hundreds of allocations from the pool, none of which require locking - destroy the pool at the end of the request, requiring one lock/unlock for the entire thing In general, that model works well: over the lifetime of a pool that handles O(n) allocation operations, we only have to do O(1) lock/unlock operations. But there's an additional optimization that's possible: Sander Striker added support for thread-specific allocators a while back. The per-request pools in most of the Apache 2.0 MPMs take advantage of this. Each pool has an apr_allocator object that supplies it with raw memory blocks. These allocators can be designated as either thread-private or shared across threads. And a thread-private allocator gets to skip the mutex operations. If you want to try the APR pools, the easiest approach might be to either use the request_rec-pool directly or create a sub-pool from it. --Brian -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performance of Apache 2.0 Filter
On Sat, Jun 08, 2002 at 02:03:21PM -0400, Cliff Woolley wrote: If the unbuffered write really *is* allowed to be buffered (wtf?), then by all means we should change to using apr_brigade_write() there instead of the flush() buckets. The data that gets passed to ub_write() is transient in lifetime so apr_brigade_write() will be forced to copy it, but that's better than sending out 11 byte packets by a long shot. Big +1 -aaron -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] filesystem security questions
The more I think about it, I dont know if streams should be done in the domxml extension currently. This is a big change from its current implementation and if it was to be undertaken, then why not just start fresh with the unified implementation? You are talking about implementing everything as streams (the dom, nodes, etc..) correct? Even with stream, how would this help in the following problem. You need to save your document to file. You may be passing it in as a stream, but when the libxml call is made it just requires a file path. validation on the file path is still going to be needed. I cant see using a stream for this as we are not doing any type of I/O here, just calling another library with a file path. I might be missing with streams, but after reading the readme, i still dont see how they will help in this situation. I feel like I got side tracked from my origional question and am back at square one. Rob Are you talking about your own extension or ext/domxml ? I think (but could be wrong) that sooner or later streams should be used everyone for consistency. - Markus -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] CVS Account Request: netholic
I want to translate docs into Korean. -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performance of Apache 2.0 Filter
At 10:47 AM 6/8/2002 -0700, Brian Pane wrote: Andi Gutmans wrote: I just checked and it seems like Apache APR memory pools use mutex locking. It'd be better to use functions like the Win32 ones which don't use mutex locking (as we made sure that only one thread allocates from its pool). This could be achieved by compiling apr_pools.c without APR_HAS_THREADS but I bet the default Apache 2 build has this enabled. Any ideas? Andi It's not as bad as it looks. :-) The APR pools do locking only in a few situations: * Creating a new pool * Creating a subpool of a pool * Destroying a pool * Adding space to an existing pool, if it's used up its initial 8KB allocation So the simplest case, the lifetime of a pool looks like: - create the pool, requiring a lock/unlock - do hundreds of allocations from the pool, none of which require locking You mean that different threads can allocate/free memory without requiring locking? How does it accomplish this? - destroy the pool at the end of the request, requiring one lock/unlock for the entire thing In general, that model works well: over the lifetime of a pool that handles O(n) allocation operations, we only have to do O(1) lock/unlock operations. But there's an additional optimization that's possible: Sander Striker added support for thread-specific allocators a while back. The per-request pools in most of the Apache 2.0 MPMs take advantage of this. Each pool has an apr_allocator object that supplies it with raw memory blocks. These allocators can be designated as either thread-private or shared across threads. And a thread-private allocator gets to skip the mutex operations. The thread-private pool is exactly what we need. That's what we're doing in IIS and it improved performance significantly. Do I have access to these pools by default? What would be the equivalent of the MS HeapCreate(HEAP_NO_SERIALIZE, ..) call? If you're interested take a look at zend_alloc.c and look for Heap. You'll see all of the relevant places. If you could work with me on this it would definitely be good for PHP. We'd just have to find a way to make it work nicely with the SAPI abstraction. It's no problem #ifdef'ing for WIN32 in zend_alloc.c but we can't do the same for SAPI modules. Andi If you want to try the APR pools, the easiest approach might be to either use the request_rec-pool directly or create a sub-pool from it. -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performanceof Apache 2.0 Filter
Andi Gutmans wrote: At 10:47 AM 6/8/2002 -0700, Brian Pane wrote: ... So the simplest case, the lifetime of a pool looks like: - create the pool, requiring a lock/unlock - do hundreds of allocations from the pool, none of which require locking You mean that different threads can allocate/free memory without requiring locking? How does it accomplish this? Each pool is only allowed to be used by one thread at a time. If the application has multiple threads that need to use the same pool, it's responsible for doing its own synchronization. But for things like the httpd, this is unnecessary: the pool that holds the data for a request is only accessed by the one thread that's handling that request. ... The thread-private pool is exactly what we need. That's what we're doing in IIS and it improved performance significantly. Do I have access to these pools by default? What would be the equivalent of the MS HeapCreate(HEAP_NO_SERIALIZE, ..) call? If you're interested take a look at zend_alloc.c and look for Heap. You'll see all of the relevant places. If you could work with me on this it would definitely be good for PHP. We'd just have to find a way to make it work nicely with the SAPI abstraction. It's no problem #ifdef'ing for WIN32 in zend_alloc.c but we can't do the same for SAPI modules. I just looked through zend_alloc.c. It looks like the HeapCreate only happens once, at startup--did I get that right? For PHP processing, do you need a memory heap that is created at startup and used by all requests, or is it sufficient to create and destroy a private heap for each request? If it's the latter, then we can probably figure out a way to have the PHP allocator grab memory from the Apache request's pool. But if it's the former (a heap used by many requests), then the problem isn't as easy to solve. --Brian -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performance of Apache 2.0 Filter
PHP has its own buffering mechanism which can take care of this. Try
output_buffering = 4096 in your php.ini.
Zeev
At 08:33 PM 6/8/2002, Brian Pane wrote:
Looking at some more syscall call traces, I'm seeing that
the flush buckets used by php_apache_sapi_ub_write() are
causing very small packets even for script-generated output,
not just for blocks of static text.
For example, I just compared the write patterns for 2.0.37
and 1.3.24, both with the default configuration for PHP-4.2.1,
while delivering a file consisting solely of:
?phpinfo() ?
1.3:
writev(3, [{HTTP/1.1 200 OK\r\nDate: Sat, 08 J..., 4089}, {28\r\n, 4},
{tr valign=\baseline\ bgcolor=\#..., 40}, {\r\n, 2}], 4) = 4135
write(3, ffb\r\ntd bgcolor=\#ff\bas..., 4098) = 4098
writev(3, [{fdd\r\ntd bgcolor=\#ff\bhig..., 4068}, {22\r\n,
4}, {/bbr //tdtd align=\center..., 34}, {\r\n, 2}], 4) = 4108
writev(3, [{ff2\r\n/usr/sbin/sendmail -t -i /..., 4089}, {22\r\n, 4},
{/bbr //tdtd align=\center..., 34}, {\r\n, 2}], 4) = 4129
(followed by a few more 4KB writes)
2.0:
writev(8, [{HTTP/1.1 200 OK\r\nDate: Sat, 08 J..., 220}, {47\r\n, 4},
{!DOCTYPE HTML PUBLIC \-//W3C//D..., 71}, {\r\n, 2}], 4) = 297
writev(8, [{6\r\n, 3}, {head, 6}, {\r\n, 2}], 3) = 11
writev(8, [{1c\r\n, 4}, { !-- DEFANGED_STYLE
type=\text/css\!--\n, 28}, {\r\n, 2}], 3) = 34
writev(8, [{1d\r\n, 4}, {a { text-decoration: none; }\n, 29}, {\r\n,
2}], 3) = 35
writev(8, [{28\r\n, 4}, {a:hover { text-decoration: under..., 40},
{\r\n, 2}], 3) = 46
writev(8, [{55\r\n, 4}, {h1 { font-family: arial, helveti..., 85},
{\r\n, 2}], 3) = 91
writev(8, [{55\r\n, 4}, {h2 { font-family: arial, helveti..., 85},
{\r\n, 2}], 3) = 91
writev(8, [{49\r\n, 4}, {body, td { font-family: arial, h..., 73},
{\r\n, 2}], 3) = 79
writev(8, [{56\r\n, 4}, {th { font-family: arial, helveti..., 86},
{\r\n, 2}], 3) = 92
writev(8, [{e\r\n, 3}, {//-- -- /DEFANGED_STYLE\n, 14}, {\r\n,
2}], 3) = 19
(followed by a thousand more tiny writes)
It looks like unbuffered PHP output in 1.3 is benefitting from the
httpd's own buffering. And unbuffered output in 2.0, in contrast,
is really, really not buffered, because of the flush buckets inserted
by php_apache_sapi_ub_write. And the tiny packets that result in the
2.0 case are likely a major cause of reports of slowness under 2.0.
--Brian
--
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php
--
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performance of Apache 2.0 Filter
At 12:55 AM 6/9/2002, Brian Pane wrote: I just looked through zend_alloc.c. It looks like the HeapCreate only happens once, at startup--did I get that right? It's called on the per-thread startup (start_memory_manager(), which is called from alloc_globals_ctor(), which is the per-thread allocation 'constructor'). For PHP processing, do you need a memory heap that is created at startup and used by all requests, or is it sufficient to create and destroy a private heap for each request? We can go either way, but won't allocating and freeing these heaps increase fragmentation? Remember that PHP functions will also allocate memory outside these heaps (in 3rd party libraries). If it's the latter, then we can probably figure out a way to have the PHP allocator grab memory from the Apache request's pool. But if it's the former (a heap used by many requests), then the problem isn't as easy to solve. We can give it a try certainly, to see if we're in the right direction. I think that in a real world situation, though, it may result in a fragmentation problem. Zeev -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performance of Apache
Cliff Woolley wrote: It looks like unbuffered PHP output in 1.3 is benefitting from the httpd's own buffering. And unbuffered output in 2.0, in contrast, is really, really not buffered, because of the flush buckets inserted by php_apache_sapi_ub_write. If the unbuffered write really *is* allowed to be buffered (wtf?), then by all means we should change to using apr_brigade_write() there instead of the flush() buckets. You say buffered, I say unbuffered, let's call the whole thing off... :) Seriously though, +1 -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performanceof Apache 2.0 Filter
Zeev Suraski wrote: PHP has its own buffering mechanism which can take care of this. Try output_buffering = 4096 in your php.ini. That works, but IMHO it won't help enough unless it becomes the default. The problem, from my perspective, is that people using the default, non-buffered setting who upgrade from Apache 1.3 to 2.0 are going to get an unpleasant surprise: their scripts that used to perform well are now a lot slower, and the load on their servers is higher. I like Cliff's suggestion of switching to apr_brigade_write() for the unbuffered case, to match the behavior of 1.3. --Brian -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performanceof Apache 2.0 Filter
Zeev Suraski wrote: At 12:55 AM 6/9/2002, Brian Pane wrote: I just looked through zend_alloc.c. It looks like the HeapCreate only happens once, at startup--did I get that right? It's called on the per-thread startup (start_memory_manager(), which is called from alloc_globals_ctor(), which is the per-thread allocation 'constructor'). For PHP processing, do you need a memory heap that is created at startup and used by all requests, or is it sufficient to create and destroy a private heap for each request? We can go either way, but won't allocating and freeing these heaps increase fragmentation? Remember that PHP functions will also allocate memory outside these heaps (in 3rd party libraries). In the httpd, we've done two things to minimize the fragmentation: * Memory for these heaps is almost always allocated in chunks of a fixed size, 8KB. * There's a two-layer structure to the heaps: - apr_pool objects are what the application uses. Each pool provides a fast alloc interface, no free function, and a destructor that returns all the allocated space when the pool is destroyed. - apr_allocator objects provide blocks of memory for the apr_pools to use. Each apr_allocator maintains a free list of blocks. When a pool is destroyed, its space is returned to the apr_allocator from which it was created. The next time we create a pool from that apr_allocator, we can take an 8KB block from that allocator's free list rather than interacting with the system heap. --Brian -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performance of Apache 2.0 Filter
At 02:57 AM 6/9/2002, Brian Pane wrote: In the httpd, we've done two things to minimize the fragmentation: * Memory for these heaps is almost always allocated in chunks of a fixed size, 8KB. Hmm, but doesn't that mean that the largest contiguous block this heap will be able to provide is 8KB, then? * There's a two-layer structure to the heaps: - apr_pool objects are what the application uses. Each pool provides a fast alloc interface, no free function, and a destructor that returns all the allocated space when the pool is destroyed. This is probably not very suitable for PHP. We allocate and free *a lot*, not being able to free is going to increase memory consumption significantly. If we use APR heaps, are we bound by this behavior? Zeev -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: PHP profiling results under 2.0.37 Re: Performance of Apache 2.0 Filter
At 04:57 PM 6/8/2002 -0700, Brian Pane wrote: Zeev Suraski wrote: At 12:55 AM 6/9/2002, Brian Pane wrote: I just looked through zend_alloc.c. It looks like the HeapCreate only happens once, at startup--did I get that right? It's called on the per-thread startup (start_memory_manager(), which is called from alloc_globals_ctor(), which is the per-thread allocation 'constructor'). For PHP processing, do you need a memory heap that is created at startup and used by all requests, or is it sufficient to create and destroy a private heap for each request? We can go either way, but won't allocating and freeing these heaps increase fragmentation? Remember that PHP functions will also allocate memory outside these heaps (in 3rd party libraries). In the httpd, we've done two things to minimize the fragmentation: * Memory for these heaps is almost always allocated in chunks of a fixed size, 8KB. * There's a two-layer structure to the heaps: - apr_pool objects are what the application uses. Each pool provides a fast alloc interface, no free function, and a destructor that returns all the allocated space when the pool is destroyed. - apr_allocator objects provide blocks of memory for the apr_pools to use. Each apr_allocator maintains a free list of blocks. When a pool is destroyed, its space is returned to the apr_allocator from which it was created. The next time we create a pool from that apr_allocator, we can take an 8KB block from that allocator's free list rather than interacting with the system heap. It sounds to me as if we're going to have to write a new allocator with the same semantics as the Win32 API one. I don't think any of these two allocators would suit us very well. Andi -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
