Re: [PHP] Persistent PHP web application?
On Sat, 08 Jan 2005 12:03:10 -0800, Rasmus Lerdorf <[EMAIL PROTECTED]> wrote: > > You greatly underestimate how slow unserialize is. > > -Rasmus > > you're right, but php-devs seems going to rewrite it. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
Xuefer Tinys wrote: even apc/eAcc have to copy from shm to php-memory(alloc by emalloc), because $array might be modified anytime i'd guess this is almost same as serialize/unserialize You greatly underestimate how slow unserialize is. -Rasmus -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
even apc/eAcc have to copy from shm to php-memory(alloc by emalloc), because $array might be modified anytime i'd guess this is almost same as serialize/unserialize i doubt if there's anyway to have 1 memcpy $array<-from/to->shm, php will release 1 element of $array when the element is unset(refcount=0) maybe apc set element refcount+1, so it won't free by php, but apc free the whole block itself? or rely on zend memory manager to free? apc opcode seems doing this way, but i don't know it much. sth off topic: the constant-array definition imho, should be optimized to which like static variable but each time get a new copy $a = array(...big..) to: static $__a = array(big...) $a = $__a; it seems static var isn't store as opcode, i'm not quite sure well, end of off topic :) On Thu, 06 Jan 2005 10:58:07 -0800, Rasmus Lerdorf <[EMAIL PROTECTED]> wrote: > Adrian Madrid wrote: > > I think I understand where you're coming from. I've had a similar > > problem and the best solution I've found is eAccelerator (previously > > known as Turck MMCache). What EA does is keep the bytecodes PHP compiles > > inshared memory so next time you need that script PHP doesn't need to > > recompile, EA returns the bytecode from SHM. Now, since PHP scripts are > > compiled and saved in SHM all I need to do is /save/ the data that does > > not change often but requires a lot of queries as code (an array inside > > a script) and include it whenever I need the data. No recompiling, no > > need to touch the DB again, pure speed. I hope all this helps you. I > > personally don't need extra processes and stuff like that but if you > > really want all that you can take a look at phpbeans. > > What you are talking about is opcode caching. While it certainly speeds > things up, it can be done much faster. When you cache a file that > contains a large PHP array definition, all you are caching are the > instructions to create that array. On every request these instructions > need to be loaded from shared memory and executed in order to recreate > the array. This can be quite slow. What we are discussing here are > ways to avoid recreating the array on every request which is quite > different. > > -Rasmus > > -- > PHP General Mailing List (http://www.php.net/) > To unsubscribe, visit: http://www.php.net/unsub.php > > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] Persistent PHP web application?
I wrote a PHP extension a while ago that implements "executor" persistence
for scalar variables and constants. I never looked much into persisting
objects, arrays or resources but it would be a useful addition to the
extension if someone wants to contribute. I haven't updated the Web site
with the latest version that compiles under 4.3.x, but if you're interested
I can send you the files that changed in the meantime. It has been immensely
useful for my projects.
http://pwee.sourceforge.net
Lance
-Original Message-
From: Josh Whiting [mailto:[EMAIL PROTECTED]
Sent: Monday, January 03, 2005 11:28 AM
To: php-general@lists.php.net
Subject: [PHP] Persistent PHP web application?
Dear list,
My web application (an online classifieds server) requires a set of
fairly large global arrays which contain vital information that most all
the page scripts rely upon for information such as the category list,
which fields belong to each category, and so on. Additionally, there are
a large number of function definitions (more than 13,000 lines of code
in all just for these global definitions).
These global arrays and functions never change between requests.
However, the PHP engine destroys and recreates them every time. After
having spent some serious time doing benchmarking (using Apache Bench),
I have found that this code takes at least 7ms to parse per request on
my dual Xeon 2.4ghz server (Zend Accelerator in use*). This seriously
cuts into my server's peak capacity, reducing it by more than half.
My question is: is there a way to define a global set of variables and
functions ONCE per Apache process, allowing each incoming hit to run a
handler function that runs within a persistent namespace? OR, is it
possible to create some form of shared variable and function namespace
that each script can tap?
AFAIK, mod_python, mod_perl, Java, etc. all allow you to create a
persistent, long-running application with hooks/handlers for individual
Apache requests. I'm surprised I haven't found a similar solution for
PHP.
In fact, according to my work in the past few days, if an application
has a large set of global functions and variable definitions, mod_python
FAR exceeds the performance of mod_php, even though Python code runs
significantly slower than PHP code (because in mod_python you can put
all these definitions in a module that is loaded only once per Apache
process).
The most promising prospect I've come across is FastCGI, which for Perl
and other languages, allows you to run a while loop that sits and
receives incoming requests (e.g. "while(FCGI::accept() >= 0) {..}").
However, the PHP/FastCGI modality seems to basically compare to mod_php:
every request still creates and destroys the entire application
(although the PHP interpreter itself does persist).
Essentially I want to go beyond a persistent PHP *interpreter* (mod_php,
PHP/FastCGI) and create a persistent PHP *application*... any
suggestions?
Thanks in advance for any help!
Regards,
J. Whiting
* - Please note that I am using the Zend Accelerator (on Redhat
Enterprise with Apache 1.3) to cache the intermediate compiled PHP code.
My benchmarks (7ms+) are after the dramatic speedup provided by the
accelerator. I wouldn't even bother benchmarking this without the
compiler cache, but it is clear that a compiler cache does not prevent
PHP from still having to run the (ableit precompiled) array and function
definition code itself.
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
On Thu, Jan 06, 2005 at 08:41:57PM -0500, Jason Barnett wrote: > >Does "not up to date" mean the code isn't working with current releases > >of php 4 or 5? I'd be interested in giving it a try. > > I believe this is the case. AFAIK the APC library doesn't support PHP5 > or at least it didn't when I looked at it. If you want to pitch in > support for APC you should just download it (or PEAR INSTALL it from the > command line to try it :) ) I don't think Rasmus was talking about APC. AFAIK he mentioned some extension code that used the Apache SAPI to run a PHP script to acheive what we're talking about (persistent global vars/functions/objects etc). (quoting Rasmus): > The apache-hooks SAPI can do this, although when I initially wrote it > nobody seemed interested. George fixed it/rewrote it to work much better, > but nobody was interested in his version either, so the current > implementation isn't up to date anymore. The problem was likely that > people really didn't understand how to use it and it is a rather unsafe > thing to let PHP scripts fiddle with all the various Apache request hooks. > You can really toast your server that way. A simple generic data loading > extension is something people will be able to understand and they are > unlikely to destroy their server with it. Rasmus, can you say more about what this is and it's current state of functionality (or lack thereof)? (back to quoting Jason): > >I'm picturing an extension that would simply not allow an uncautious or > >unknowing scripter to ruin the server, but would only allow him to store > >all his app defs and data model startup stuff in a one-shot performance > >booster. It could even do it in a separate (CLI?) interpreter and copy > >the data over (but only once!) to the Apache-embedded interpreter using > >a shared memory resource... hmmm. > > So your process is something like (correct me if I misunderstand): > [snip...] > The above suggestion is quite messy and probably not thread-safe, but > it's a start. ;) Well, bravo for venturing a possible implementation, but I'm not versed in Apache or PHP internals to propose or evaluate one in that kind of detail :(. I don't know what is possible or how PHP internally stores variables, functions, objects, etc that would make them able to persist or be shuffled around. /jw -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
On Fri, 7 Jan 2005, William Lovaton wrote: > The phpbeans (or sockets) kind of solutions won't work as Josh Whiting > (original author of this thread) would expect. > > phpbeans is a good idea to share (complex) business logic and data > between many web servers but it will have to serialize and unserialize > the data to send it across the network and it will kill the purpose of > this discussion. > > See the post from Rasmus who says that APC package is able to store data > in a shared memory segment without serialization. It still does a > mem_cpy() though. This would be a real solution for the problem, but > there are some others. > http://pecl.php.net/package/APC > > Rasmus, you said this feature is currently in CVS... any idea about a > new release? What kind of features offers the currently stable version > (2.0.4) in this regard? 2.0.4 does not support this. I have a patch I am testing to fix some other issues and once I get that working I will probably push out a new release. > Do you know where can I look for performance comparison between APC and > other opcode cache systems like Turck, Zend Accel or eA?? These days they are all in the same ballpark. The optimizer in APC is not very strong so it loses a percent or two there, but on the other hand many people turn off the optimizer in other caches because it sometimes screws up their scripts. -Rasmus -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
Hi everybody in this thread, The phpbeans (or sockets) kind of solutions won't work as Josh Whiting (original author of this thread) would expect. phpbeans is a good idea to share (complex) business logic and data between many web servers but it will have to serialize and unserialize the data to send it across the network and it will kill the purpose of this discussion. See the post from Rasmus who says that APC package is able to store data in a shared memory segment without serialization. It still does a mem_cpy() though. This would be a real solution for the problem, but there are some others. http://pecl.php.net/package/APC Rasmus, you said this feature is currently in CVS... any idea about a new release? What kind of features offers the currently stable version (2.0.4) in this regard? Do you know where can I look for performance comparison between APC and other opcode cache systems like Turck, Zend Accel or eA?? Thanx to all, -William El jue, 06-01-2005 a las 13:32 -0700, Adrian Madrid escribió: > I've done some benchmaring and it is quite fast, specially compared to > talking to the DB. Also, phpbeans provides a daemon that can give you > the solution you are looking for, I believe. > > Adrian > > > Rasmus Lerdorf wrote: > > > Adrian Madrid wrote: > > > >> I think I understand where you're coming from. I've had a similar > >> problem and the best solution I've found is eAccelerator (previously > >> known as Turck MMCache). What EA does is keep the bytecodes PHP > >> compiles inshared memory so next time you need that script PHP > >> doesn't need to recompile, EA returns the bytecode from SHM. Now, > >> since PHP scripts are compiled and saved in SHM all I need to do is > >> /save/ the data that does not change often but requires a lot of > >> queries as code (an array inside a script) and include it whenever I > >> need the data. No recompiling, no need to touch the DB again, pure > >> speed. I hope all this helps you. I personally don't need extra > >> processes and stuff like that but if you really want all that you can > >> take a look at phpbeans. > > > > > > What you are talking about is opcode caching. While it certainly > > speeds things up, it can be done much faster. When you cache a file > > that contains a large PHP array definition, all you are caching are > > the instructions to create that array. On every request these > > instructions need to be loaded from shared memory and executed in > > order to recreate the array. This can be quite slow. What we are > > discussing here are ways to avoid recreating the array on every > > request which is quite different. > > > > -Rasmus > > > > > -- > Adrian Madrid > HyperX Inc. > Mobile: 801.815.1870 > Office: 801.566.0670 > [EMAIL PROTECTED] > www.hyperxmedia.com > > 9000 S. 45 W. > Sandy, UT 84070 > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
> > Call me crazy or ignorant, i'm both, but would it be possible to build > > an extension that, in its MINIT hook as you suggest, actually runs a > > separate PHP script that contains global definitions, then makes those > > definitions available to later scripts? this is basically my original > > desire of having a one-time, persistent global include for each apache > > process. i realize you suggest pulling array data from a source like a > > DB or XML file, which would be a 90% solution for me, but the next step > > (a PHP script) just seemed logical/exciting... > > > > i realize i'm reaching with this, and it implies a conundrum (how does > > an extension run a PHP script if PHP hasn't fully started yet) but this > > kind of thing is just what makes me go and learn new things (C, php/zend > > internals) and I just might try to do it if it was feasible, for the fun > > of it, because it could be useful to lots of people, even if it took me > > a year or two to pull off. > > > > alas, this is a question for the pecl-dev list. > > > > (the mental gears are churning...) > > The apache-hooks SAPI can do this, although when I initially wrote it > nobody seemed interested. George fixed it/rewrote it to work much better, > but nobody was interested in his version either, so the current > implementation isn't up to date anymore. The problem was likely that > people really didn't understand how to use it and it is a rather unsafe > thing to let PHP scripts fiddle with all the various Apache request hooks. > You can really toast your server that way. A simple generic data loading > extension is something people will be able to understand and they are > unlikely to destroy their server with it. > > -Rasmus Does "not up to date" mean the code isn't working with current releases of php 4 or 5? I'd be interested in giving it a try. Forgive my ignorance of Apache and PHP internals, but I'm not understanding the concept of your implementation. I'm not enivisioning allowing a PHP script to be able to meddle with the Apache request process. I'm picturing an extension that would simply not allow an uncautious or unknowing scripter to ruin the server, but would only allow him to store all his app defs and data model startup stuff in a one-shot performance booster. It could even do it in a separate (CLI?) interpreter and copy the data over (but only once!) to the Apache-embedded interpreter using a shared memory resource... hmmm. I'm not versed enough to suggest a feasible implementation, but if the overall goal is feasible then I'm willing to climb the learning curve. I'm also quite surprised there wasn't much interest in these concepts, I admit the benefits would be small for most apps, but complex apps with large data setup could save valuable time, and having persistent namespaces/modules is plugged as a worthy feature of mod_perl and mod_python, for what it's worth. /jw -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
Does "not up to date" mean the code isn't working with current releases of php 4 or 5? I'd be interested in giving it a try. I believe this is the case. AFAIK the APC library doesn't support PHP5 or at least it didn't when I looked at it. If you want to pitch in support for APC you should just download it (or PEAR INSTALL it from the command line to try it :) ) Forgive my ignorance of Apache and PHP internals, but I'm not understanding the concept of your implementation. I'm not enivisioning allowing a PHP script to be able to meddle with the Apache request process. I'm picturing an extension that would simply not allow an uncautious or unknowing scripter to ruin the server, but would only allow him to store all his app defs and data model startup stuff in a one-shot performance booster. It could even do it in a separate (CLI?) interpreter and copy the data over (but only once!) to the Apache-embedded interpreter using a shared memory resource... hmmm. So your process is something like (correct me if I misunderstand): - have a PHP script with all global variables and class definitions that will be available to *all* scripts - parse the PHP file to generate default values for variables - as a side note: would this include any autoglobals such as $_SERVER, $_ENV, etc. ? I would think not at least for our first version. - have a function translate all object definitions into the binary version of the definition and return that definition as a string. - have a function translate each PHP variable into the binary value and return these as a string as well. - write all of these strings to some file (test.static) Restart server. Here is where I get really fuzzy, mostly because I have never written an extension. ;) - During MINIT - Lock file (test.static) for reading - We read from the source file (test.static) - We initialize the non-object variables in a PHP-usable form - We *try* to initialize objects. Likely to be some issues here, for example, when a class definition changes. - As a side note, should we create a superglobal array to store these global variables similar to $_SESSION? Maybe $_SHARED? It is likely a pain to implement, but would be cool. :) - Module needs a fetch function to get the value in shared memory. - It needs to get a "lock" to read to that part of shared memory (forgive me but I don't know the right term here) - For objects, this needs to check access level (private, protected, public) and return appropriate errors - Module needs a write function to write to shared memory. - It needs to get a "lock" to write to that part of shared memory - For objects, this needs to return appropriate errors for modifying properties (i.e. private, protected, public) and return appropriate errors - During shutdown (I think it's called MSHUTDOWN) - Lock file (test.static) for writing - Recreate the test.static file The above suggestion is quite messy and probably not thread-safe, but it's a start. ;) -- Teach a person to fish... Ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html PHP Manual: http://php.net/manual/ php-general archives: http://marc.theaimsgroup.com/?l=php-general&w=2 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
> I think I understand where you're coming from. I've had a similar > problem and the best solution I've found is eAccelerator (previously > known as Turck MMCache). What EA does is keep the bytecodes PHP compiles > inshared memory so next time you need that script PHP doesn't need to > recompile, EA returns the bytecode from SHM. Now, since PHP scripts are > compiled and saved in SHM all I need to do is /save/ the data that does > not change often but requires a lot of queries as code (an array inside > a script) and include it whenever I need the data. No recompiling, no > need to touch the DB again, pure speed. I hope all this helps you. I > personally don't need extra processes and stuff like that but if you > really want all that you can take a look at phpbeans. > > Hope it helps, > > > Adrian Madrid phpBeans looks interesting. after browsing the site a bit, looking at the introductory material and so on, i couldn't discern the way the phpbean server persists the beans, and, hence, if the solution would be fitting for what i'm looking for. what i'm looking for is a solution that will keep compiled zend variables and functions (the end result of executed php code that defines the variables and functions), in memory, without serialization, across multiple requests. if this is how the beans are kept persistent, i'm interested :) i'm going to try to find someone at phpbeans who can answer that. if others are interested, email me and i'll let you know what i find (if/when i find it)... /jw -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
http://eaccelerator.sourceforge.net/Home Go there, you'll find more about it. In short, Zend hired the lead programmer of MMCache and the project eventually died off. Other people picked it up and developed patches to update it. I believe eAccelerator is the most advanced and stable for me in PHP 5. I'm running it and it's been good to me. Although I recommend taking the optimizer off. Anyway, hope it helped. Adrian Madrid William Lovaton wrote: El jue, 06-01-2005 a las 10:26 -0700, Adrian Madrid escribió: I think I understand where you're coming from. I've had a similar problem and the best solution I've found is eAccelerator (previously known as Turck MMCache). Hold on a second! I use Turck MMCache but I didn't know about the change of name... what is the history behind this? What is the state of development? -William -- Adrian Madrid HyperX Inc. Mobile: 801.815.1870 Office: 801.566.0670 [EMAIL PROTECTED] www.hyperxmedia.com 9000 S. 45 W. Sandy, UT 84070
Re: [PHP] Persistent PHP web application?
> I've done some benchmaring and it is quite fast, specially compared to > talking to the DB. Also, phpbeans provides a daemon that can give you > the solution you are looking for, I believe. > > Adrian Hey I just checked out phpbeans, it looks pretty intense, it does have a php deamon that runs asthe server and the client connects to it via RMI based possible socket ? connections. It also lookslike you reference the object you want to display on the page like web-app/contact where contactis the possible class. This looks pretty interesting because i have currently ported alot of my codeto classes and still need a php script per page to execute the class via the constructor, this couldsimplify this process although i just checked how they add objects and i dont see any sharedmemory or session management for the objects ? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
>>
>>>From my PHP library I use shm_put_var() and shm_get_var(). If
>> serialization is done this way then it is implicit... right?
>
> Yes, these functions serialize/unserialize behing the scenes.
>
> -Rasmus
Hi, has anyone got an example on howto use the shared memory functions ? I
currently serializeobjects of classes so they are cache and dont have to be
included upon
each request, i store thesein a session , its in a function called import heheh
, so
import('Package_Class'); , will get a class ina directory Package/Class.php
with the class name Package_Class , and
store that in a session.
would it be a quicker solution when unserializing to use the shared memory
functions instead ?
Let me know
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
Hello, on 01/06/2005 05:24 PM William Lovaton said the following: El jue, 06-01-2005 a las 10:26 -0700, Adrian Madrid escribió: I think I understand where you're coming from. I've had a similar problem and the best solution I've found is eAccelerator (previously known as Turck MMCache). Hold on a second! I use Turck MMCache but I didn't know about the change of name... what is the history behind this? What is the state of development? It is a fork of Turck continued by other developers, as you cannot expect Turck author to continue developing a competing product to his new employer Zend! -- Regards, Manuel Lemos PHP Classes - Free ready to use OOP components written in PHP http://www.phpclasses.org/ PHP Reviews - Reviews of PHP books and other products http://www.phpclasses.org/reviews/ Metastorage - Data object relational mapping layer generator http://www.meta-language.net/metastorage.html -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
El jue, 06-01-2005 a las 10:52 -0800, Rasmus Lerdorf escribió: > William Lovaton wrote: > >From my PHP library I use shm_put_var() and shm_get_var(). If > > serialization is done this way then it is implicit... right? > > Yes, these functions serialize/unserialize behing the scenes. Rasmus, I am using Turck MMCache in my production server only as an op code cache and I even developed a backed in my library to use Turck's APIs to cache data in it. However I had to go back to SHM because I had some random problems with it that I couldn't solve. I wonder if Turck doesn't serialize/unserialize just like APC does, May be you know. -William -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
I've done some benchmaring and it is quite fast, specially compared to talking to the DB. Also, phpbeans provides a daemon that can give you the solution you are looking for, I believe. Adrian Rasmus Lerdorf wrote: Adrian Madrid wrote: I think I understand where you're coming from. I've had a similar problem and the best solution I've found is eAccelerator (previously known as Turck MMCache). What EA does is keep the bytecodes PHP compiles inshared memory so next time you need that script PHP doesn't need to recompile, EA returns the bytecode from SHM. Now, since PHP scripts are compiled and saved in SHM all I need to do is /save/ the data that does not change often but requires a lot of queries as code (an array inside a script) and include it whenever I need the data. No recompiling, no need to touch the DB again, pure speed. I hope all this helps you. I personally don't need extra processes and stuff like that but if you really want all that you can take a look at phpbeans. What you are talking about is opcode caching. While it certainly speeds things up, it can be done much faster. When you cache a file that contains a large PHP array definition, all you are caching are the instructions to create that array. On every request these instructions need to be loaded from shared memory and executed in order to recreate the array. This can be quite slow. What we are discussing here are ways to avoid recreating the array on every request which is quite different. -Rasmus -- Adrian Madrid HyperX Inc. Mobile: 801.815.1870 Office: 801.566.0670 [EMAIL PROTECTED] www.hyperxmedia.com 9000 S. 45 W. Sandy, UT 84070 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
El jue, 06-01-2005 a las 10:26 -0700, Adrian Madrid escribió: > I think I understand where you're coming from. I've had a similar > problem and the best solution I've found is eAccelerator (previously > known as Turck MMCache). Hold on a second! I use Turck MMCache but I didn't know about the change of name... what is the history behind this? What is the state of development? -William -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
What you are talking about is opcode caching. While it certainly speeds things up, it can be done much faster. When you cache a file that contains a large PHP array definition, all you are caching are the instructions to create that array. On every request these instructions need to be loaded from shared memory and executed in order to recreate the array. This can be quite slow. What we are discussing here are ways to avoid recreating the array on every request which is quite different. -Rasmus Wow. I've learned things in this newsgroup, but today's discussion has been particularly enlightening. Thank you professors Rasmus, Richard and William. :) -- Teach a person to fish... Ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html PHP Manual: http://php.net/manual/ php-general archives: http://marc.theaimsgroup.com/?l=php-general&w=2 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
Adrian Madrid wrote: I think I understand where you're coming from. I've had a similar problem and the best solution I've found is eAccelerator (previously known as Turck MMCache). What EA does is keep the bytecodes PHP compiles inshared memory so next time you need that script PHP doesn't need to recompile, EA returns the bytecode from SHM. Now, since PHP scripts are compiled and saved in SHM all I need to do is /save/ the data that does not change often but requires a lot of queries as code (an array inside a script) and include it whenever I need the data. No recompiling, no need to touch the DB again, pure speed. I hope all this helps you. I personally don't need extra processes and stuff like that but if you really want all that you can take a look at phpbeans. What you are talking about is opcode caching. While it certainly speeds things up, it can be done much faster. When you cache a file that contains a large PHP array definition, all you are caching are the instructions to create that array. On every request these instructions need to be loaded from shared memory and executed in order to recreate the array. This can be quite slow. What we are discussing here are ways to avoid recreating the array on every request which is quite different. -Rasmus -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
William Lovaton wrote: Rasmus, El jue, 06-01-2005 a las 08:23 -0800, Rasmus Lerdorf escribió: On Thu, 6 Jan 2005, William Lovaton wrote: This is great. In my high performance web app I created a PHP library that abstracted this to use several backends. For instance I have a File backend and a SHM backend that uses the functions provided by the sysvshm PHP module. With this functions, do they need to serialize/unserialize every time I put/get something in/from the cache?? If you are doing this in user-space PHP, then yes, you will have to serialize. From my PHP library I use shm_put_var() and shm_get_var(). If serialization is done this way then it is implicit... right? Yes, these functions serialize/unserialize behing the scenes. -Rasmus -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
I think I understand where you're coming from. I've had a similar
problem and the best solution I've found is eAccelerator (previously
known as Turck MMCache). What EA does is keep the bytecodes PHP compiles
inshared memory so next time you need that script PHP doesn't need to
recompile, EA returns the bytecode from SHM. Now, since PHP scripts are
compiled and saved in SHM all I need to do is /save/ the data that does
not change often but requires a lot of queries as code (an array inside
a script) and include it whenever I need the data. No recompiling, no
need to touch the DB again, pure speed. I hope all this helps you. I
personally don't need extra processes and stuff like that but if you
really want all that you can take a look at phpbeans.
Hope it helps,
Adrian Madrid
eAccelerator: http://eaccelerator.sourceforge.net/Home
phpbeans: http://www.simian.ca/index/phpbeans
Josh Whiting wrote:
Dear list,
My web application (an online classifieds server) requires a set of
fairly large global arrays which contain vital information that most all
the page scripts rely upon for information such as the category list,
which fields belong to each category, and so on. Additionally, there are
a large number of function definitions (more than 13,000 lines of code
in all just for these global definitions).
These global arrays and functions never change between requests.
However, the PHP engine destroys and recreates them every time. After
having spent some serious time doing benchmarking (using Apache Bench),
I have found that this code takes at least 7ms to parse per request on
my dual Xeon 2.4ghz server (Zend Accelerator in use*). This seriously
cuts into my server's peak capacity, reducing it by more than half.
My question is: is there a way to define a global set of variables and
functions ONCE per Apache process, allowing each incoming hit to run a
handler function that runs within a persistent namespace? OR, is it
possible to create some form of shared variable and function namespace
that each script can tap?
AFAIK, mod_python, mod_perl, Java, etc. all allow you to create a
persistent, long-running application with hooks/handlers for individual
Apache requests. I'm surprised I haven't found a similar solution for
PHP.
In fact, according to my work in the past few days, if an application
has a large set of global functions and variable definitions, mod_python
FAR exceeds the performance of mod_php, even though Python code runs
significantly slower than PHP code (because in mod_python you can put
all these definitions in a module that is loaded only once per Apache
process).
The most promising prospect I've come across is FastCGI, which for Perl
and other languages, allows you to run a while loop that sits and
receives incoming requests (e.g. "while(FCGI::accept() >= 0) {..}").
However, the PHP/FastCGI modality seems to basically compare to mod_php:
every request still creates and destroys the entire application
(although the PHP interpreter itself does persist).
Essentially I want to go beyond a persistent PHP *interpreter* (mod_php,
PHP/FastCGI) and create a persistent PHP *application*... any
suggestions?
Thanks in advance for any help!
Regards,
J. Whiting
* - Please note that I am using the Zend Accelerator (on Redhat
Enterprise with Apache 1.3) to cache the intermediate compiled PHP code.
My benchmarks (7ms+) are after the dramatic speedup provided by the
accelerator. I wouldn't even bother benchmarking this without the
compiler cache, but it is clear that a compiler cache does not prevent
PHP from still having to run the (ableit precompiled) array and function
definition code itself.
--
Adrian Madrid
HyperX Inc.
Mobile: 801.815.1870
Office: 801.566.0670
[EMAIL PROTECTED]
www.hyperxmedia.com
9000 S. 45 W.
Sandy, UT 84070
Re: [PHP] Persistent PHP web application?
On Thu, 6 Jan 2005, Josh Whiting wrote: > > Anything you do in the MINIT hook is basically free, so it would be > > trivial to load the data for the array from somewhere. Like a database, > > an xml file, etc. So you wouldn't need to hardcode a complex array > > structure in your MINIT hook, just have this generic little extension > > that creates an array (or object) from some external source. To change > > the data you would change that external source and restart your server, > > or you could write a PHP function in your extension that forced a reload > > with the caveat that each running httpd process would need to have that > > function be called since your array/object lives in each process separately. > > > > If you ask really nicely one of the folks on the pecl-dev list might > > just write this thing for you. Especially if you spec it out nicely and > > think through how it should work. > > > > -Rasmus > > Call me crazy or ignorant, i'm both, but would it be possible to build > an extension that, in its MINIT hook as you suggest, actually runs a > separate PHP script that contains global definitions, then makes those > definitions available to later scripts? this is basically my original > desire of having a one-time, persistent global include for each apache > process. i realize you suggest pulling array data from a source like a > DB or XML file, which would be a 90% solution for me, but the next step > (a PHP script) just seemed logical/exciting... > > i realize i'm reaching with this, and it implies a conundrum (how does > an extension run a PHP script if PHP hasn't fully started yet) but this > kind of thing is just what makes me go and learn new things (C, php/zend > internals) and I just might try to do it if it was feasible, for the fun > of it, because it could be useful to lots of people, even if it took me > a year or two to pull off. > > alas, this is a question for the pecl-dev list. > > (the mental gears are churning...) The apache-hooks SAPI can do this, although when I initially wrote it nobody seemed interested. George fixed it/rewrote it to work much better, but nobody was interested in his version either, so the current implementation isn't up to date anymore. The problem was likely that people really didn't understand how to use it and it is a rather unsafe thing to let PHP scripts fiddle with all the various Apache request hooks. You can really toast your server that way. A simple generic data loading extension is something people will be able to understand and they are unlikely to destroy their server with it. -Rasmus -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
On Thu, 6 Jan 2005, William Lovaton wrote: > Hi Rasmus, > > El lun, 03-01-2005 a las 14:13 -0500, Rasmus Lerdorf escribió: > > If you need to do something fancier you can stick things in shared > > memory. Many of the accelerators give you access to their shared memory > > segments. For example, the CVS version of pecl/apc provides apc_store() > > and apc_fetch() which lets you store PHP datatypes in shared memory > > directly without needing to serialize/unserialize them. > > This is great. In my high performance web app I created a PHP library > that abstracted this to use several backends. For instance I have a > File backend and a SHM backend that uses the functions provided by the > sysvshm PHP module. With this functions, do they need to > serialize/unserialize every time I put/get something in/from the cache?? If you are doing this in user-space PHP, then yes, you will have to serialize. > Also I use sysvsem for locking capabilities. does apc take care of the > locking? does it have an API to do that? APC takes care of locking and all cache management. > My experience with shared memory have been great. My production server > uses a segment of 384MB to cache lots of resultsets and PHP objects and > the performance is almost unbeatable... well if it serializes then > pecl/apc will give better performance I guess. It should, yes. Try it. -Rasmus -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
Rasmus, El jue, 06-01-2005 a las 08:23 -0800, Rasmus Lerdorf escribió: > On Thu, 6 Jan 2005, William Lovaton wrote: > > This is great. In my high performance web app I created a PHP library > > that abstracted this to use several backends. For instance I have a > > File backend and a SHM backend that uses the functions provided by the > > sysvshm PHP module. With this functions, do they need to > > serialize/unserialize every time I put/get something in/from the cache?? > > If you are doing this in user-space PHP, then yes, you will have to > serialize. >From my PHP library I use shm_put_var() and shm_get_var(). If serialization is done this way then it is implicit... right? -William -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
> Anything you do in the MINIT hook is basically free, so it would be > trivial to load the data for the array from somewhere. Like a database, > an xml file, etc. So you wouldn't need to hardcode a complex array > structure in your MINIT hook, just have this generic little extension > that creates an array (or object) from some external source. To change > the data you would change that external source and restart your server, > or you could write a PHP function in your extension that forced a reload > with the caveat that each running httpd process would need to have that > function be called since your array/object lives in each process separately. > > If you ask really nicely one of the folks on the pecl-dev list might > just write this thing for you. Especially if you spec it out nicely and > think through how it should work. > > -Rasmus Call me crazy or ignorant, i'm both, but would it be possible to build an extension that, in its MINIT hook as you suggest, actually runs a separate PHP script that contains global definitions, then makes those definitions available to later scripts? this is basically my original desire of having a one-time, persistent global include for each apache process. i realize you suggest pulling array data from a source like a DB or XML file, which would be a 90% solution for me, but the next step (a PHP script) just seemed logical/exciting... i realize i'm reaching with this, and it implies a conundrum (how does an extension run a PHP script if PHP hasn't fully started yet) but this kind of thing is just what makes me go and learn new things (C, php/zend internals) and I just might try to do it if it was feasible, for the fun of it, because it could be useful to lots of people, even if it took me a year or two to pull off. alas, this is a question for the pecl-dev list. (the mental gears are churning...) /jw -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
Hi Rasmus, El lun, 03-01-2005 a las 14:13 -0500, Rasmus Lerdorf escribió: > If you need to do something fancier you can stick things in shared > memory. Many of the accelerators give you access to their shared memory > segments. For example, the CVS version of pecl/apc provides apc_store() > and apc_fetch() which lets you store PHP datatypes in shared memory > directly without needing to serialize/unserialize them. This is great. In my high performance web app I created a PHP library that abstracted this to use several backends. For instance I have a File backend and a SHM backend that uses the functions provided by the sysvshm PHP module. With this functions, do they need to serialize/unserialize every time I put/get something in/from the cache?? Also I use sysvsem for locking capabilities. does apc take care of the locking? does it have an API to do that? My experience with shared memory have been great. My production server uses a segment of 384MB to cache lots of resultsets and PHP objects and the performance is almost unbeatable... well if it serializes then pecl/apc will give better performance I guess. -William -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
Richard Lynch wrote: Or you could pay a guy who knows C and PHP to do it in, what, a couple hours? Depends on how confusing your arrays are, and how heterogeneous they are, I guess. Once you do that, all your data is in PHP/Apache when it launches, and it's always available to your PHP script all the time. Sweet. Anything you do in the MINIT hook is basically free, so it would be trivial to load the data for the array from somewhere. Like a database, an xml file, etc. So you wouldn't need to hardcode a complex array structure in your MINIT hook, just have this generic little extension that creates an array (or object) from some external source. To change the data you would change that external source and restart your server, or you could write a PHP function in your extension that forced a reload with the caveat that each running httpd process would need to have that function be called since your array/object lives in each process separately. If you ask really nicely one of the folks on the pecl-dev list might just write this thing for you. Especially if you spec it out nicely and think through how it should work. -Rasmus -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
>> So if what your application mostly does is load in all this data and >> respond to requests, you could write a *SINGLE* PHP application which >> listened on port 12345 (or whatever port you like) and responded with >> the >> data requested. Like writing your own web-server, only it's a >> _-server where you get to fill in the blank with whatever your >> application does. > > Please see my repsonse to Manuel Lemos and his suggestion to run a SOAP > server. Basically my concern is the lack of having a > multi-process/forking server to handle concurrent incoming requests > (which may or may not be a problem - not sure). We're talking about a > persistent PHP server (SOAP or otherwise), and I'm having trouble > groking how that would work in an environment with many concurrent > requests. (Other than, of course, running a PHP SOAP server inside > Apache which brings me back to square one :) [shrug] In my Dream World, you could run as many "%> php -q wyw_server.php" processes as you like, and the low-level socket code would take care of only having one of them respond to any given request. Assuming that's not Reality Based, I guess you could try to figure out how Apache does its pre-fork and child management... Ya know, Rasmus' answer of just making your own PHP Module is looking REAL GOOD right now... You'll have to stumble your way through a LITTLE bit of C -- most of which is going to be copy&paste from his example code (http://talks.php.net) And then you're going to copy&paste your monster array into your new C code. Then you'll have to change every line with $foo to just have foo, and declare foo correctly, and segregate all your data/arrays so all the arrays have only one kind of data in them, and... It's going to be confusing as hell at first, but do it with a very SMALL version of your monster array, and after you get the hang of that, it will be a lot of global Search&Replace to make your big array work. Or you could pay a guy who knows C and PHP to do it in, what, a couple hours? Depends on how confusing your arrays are, and how heterogeneous they are, I guess. Once you do that, all your data is in PHP/Apache when it launches, and it's always available to your PHP script all the time. Sweet. -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
Thanks for taking the time for your comprehensive repsonse! > However, given your programming philosophy so far, and the fact that you > are worried about 7ms and that you specifically requested some kind of > shared memory space in PHP, you should Read This: > http://us4.php.net/manual/en/ref.sem.php > > That pretty much is exactly what you asked for. > > Be forewarned that few PHP scripters have needed this stuff, and it's not > anywhere near as hammered on (read: debugged) as the other options above. I had the thought of using shared memory but I've found a scarce supply of introductory material and examples on the subject. The documentation assumes a level of familiarity with shared memory that I don't have, so I'm struggling. Examples of how to store and retreive a set of large multidimensional arrays to/from shared memory would be wonderful to look at, if you have some pointers to articles/books/etc I am also curious if the the shared memory solution would be a significant performance improvement, since, AFAIK, the data is serialized/unserialized for storage/retreival. Since the Zend Accelerator already uses a shared memory cache for the intermediate code, would using my own shared memory be any different? Consider that my global variable definition scripts are nothing more than giant unserialize statements. (Incidentally, I've benchmarked the difference between big unserialize statements and big PHP code array definition statements and found almost no difference, only a marginal improvement in performance using PHP code instead of unserialize on long strings.) > > Additionally, there are > > a large number of function definitions (more than 13,000 lines of code > > in all just for these global definitions). > > You should look into breaking these up into groups of functionality -- I'm > willing to bet you could segment these and have many pages that only call > in a few functions. That is good practice, I know i've been a bit lazy in that respect :) However, it is the variable definitions (big arrays) that take the most significant time, executing all the function definition code is actually only a small fraction of the time taken to parse the arrays, so it's a secondary concern, though you are right I should sit down and rework my includes so I'm only bringing in what I need. I'll set aside a week for that :) > So if what your application mostly does is load in all this data and > respond to requests, you could write a *SINGLE* PHP application which > listened on port 12345 (or whatever port you like) and responded with the > data requested. Like writing your own web-server, only it's a > _-server where you get to fill in the blank with whatever your > application does. Please see my repsonse to Manuel Lemos and his suggestion to run a SOAP server. Basically my concern is the lack of having a multi-process/forking server to handle concurrent incoming requests (which may or may not be a problem - not sure). We're talking about a persistent PHP server (SOAP or otherwise), and I'm having trouble groking how that would work in an environment with many concurrent requests. (Other than, of course, running a PHP SOAP server inside Apache which brings me back to square one :) > Actually, Zend and others could be interested in your comparisons of > "with" and "without" cache... If folks are interested in my benchmarks I could spend some time and put together a summary. Incidentally I did end up doing some comparison of cached code and uncached code because I unwittingly had the accelerator turned off for while until I realized it :) > Hope that helps... Almost feel like I ought to invoice you at this point :-) :) Thanks for the pro bono consulting, you have indeed helped! Regards, /josh w. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
Josh Whiting wrote:
Wow thanks for the helpful breakdown.
PHP's model is to be completely sandboxed such that every request is
completely separate from every other. Having a persistent interpreter
as you describe would break that rule and break the infinite horizontal
scalability model of PHP.
Understood. A persistent interpreter is a whole different approach. I
appreciate your perspective on that, it helps me to reconsider overall
what I'm doing and why I want to do it :)
Of course, there is nothing that prevents you from storing persistent
data somewhere more permanent. If it is just simple read-only data you
have a lot of options. For example, you could put them in a .ini file
that is only loaded on Apache startup and use get_cfg_var() to fetch
them. If you compile PHP with the --with-config-file-scan-dir switch to
configure a configuration scan directory you can just drop your own ini
file in that directory and it will be read on startup. This is just
key=value pairs and not PHP code, of course.
I'm dealing with large multidimensional arrays, like this:
$categories = array (
1 => array (
'name' => 'Autos',
'depth' => 1,
...
:(
If you need to do something fancier you can stick things in shared
memory. Many of the accelerators give you access to their shared memory
segments. For example, the CVS version of pecl/apc provides apc_store()
and apc_fetch() which lets you store PHP datatypes in shared memory
directly without needing to serialize/unserialize them.
That pecl/apc feature sounds like a great, cheap solution to my giant
global variable definition problem, which takes the biggest single chunk
of parsing time. The key AFAICS is avoiding the (un)serialization time.
I'd love to see an example if you have one, just to show me a target to
aim for. I'm unfamiliar with C / reading C source code and with shared
memory so I'm having a tough time figuring out how to use that feature.
It's trivial. You check to see if your data is in shared memory. If it
isn't, you create it and store it there using some identifier. Like this:
$huge_array = apc_fetch('my_huge_array');
if(!$huge_array) {
$huge_array = array( ... );
apc_store('my_huge_array', $huge_array);
}
You can optionally add a 3rd argument to apc_store() which is a timeout
for the data. That means you can tell APC that the data is only valid
for 600 seconds, for example.
apc_store('my_huge_array', $huge_array, 600);
And finally, the big hammer is to write your own PHP extension. This is
a lot easier than people think and for people who are really looking for
performance this is the only way to go. You can write whatever code you
want in your MINIT hook which only gets called on server startup and in
that hook you can create whatever persistent variables you need, pull
stuff from a DB, etc. At the same time you will likely want to pull
some of your heavier business logic identified by your profiling into
the extension as well. This combination of C and PHP is extremely hard
to beat performance-wise regardless of what you compare it to.
This is something I'm VERY interested in. It is encouraging to hear that
it is easier than I expect, and I will look into it further. Based on
the responses from the list I've gotten, this seems like the most
promising "total" solution. Any outstanding books/articles on the topic,
considering I'm not a C programmer?
If you don't know any C at all it is going to be an uphill battle.
Either spend some time and learn C, or hire someone to write this for
you. When I said it was easier than people expect, I was assuming some
level of C proficiency.
-Rasmus
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
Wow thanks for the helpful breakdown. > PHP's model is to be completely sandboxed such that every request is > completely separate from every other. Having a persistent interpreter > as you describe would break that rule and break the infinite horizontal > scalability model of PHP. Understood. A persistent interpreter is a whole different approach. I appreciate your perspective on that, it helps me to reconsider overall what I'm doing and why I want to do it :) > Of course, there is nothing that prevents you from storing persistent > data somewhere more permanent. If it is just simple read-only data you > have a lot of options. For example, you could put them in a .ini file > that is only loaded on Apache startup and use get_cfg_var() to fetch > them. If you compile PHP with the --with-config-file-scan-dir switch to > configure a configuration scan directory you can just drop your own ini > file in that directory and it will be read on startup. This is just > key=value pairs and not PHP code, of course. I'm dealing with large multidimensional arrays, like this: $categories = array ( 1 => array ( 'name' => 'Autos', 'depth' => 1, ... :( > If you need to do something fancier you can stick things in shared > memory. Many of the accelerators give you access to their shared memory > segments. For example, the CVS version of pecl/apc provides apc_store() > and apc_fetch() which lets you store PHP datatypes in shared memory > directly without needing to serialize/unserialize them. That pecl/apc feature sounds like a great, cheap solution to my giant global variable definition problem, which takes the biggest single chunk of parsing time. The key AFAICS is avoiding the (un)serialization time. I'd love to see an example if you have one, just to show me a target to aim for. I'm unfamiliar with C / reading C source code and with shared memory so I'm having a tough time figuring out how to use that feature. (It doesn't solve the function definition problem but that is OK - the functions take FAR less time to parse than the global variable definitions.) > And finally, the big hammer is to write your own PHP extension. This is > a lot easier than people think and for people who are really looking for > performance this is the only way to go. You can write whatever code you > want in your MINIT hook which only gets called on server startup and in > that hook you can create whatever persistent variables you need, pull > stuff from a DB, etc. At the same time you will likely want to pull > some of your heavier business logic identified by your profiling into > the extension as well. This combination of C and PHP is extremely hard > to beat performance-wise regardless of what you compare it to. This is something I'm VERY interested in. It is encouraging to hear that it is easier than I expect, and I will look into it further. Based on the responses from the list I've gotten, this seems like the most promising "total" solution. Any outstanding books/articles on the topic, considering I'm not a C programmer? Thanks again /josh w. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
Josh Whiting wrote:
Dear list,
My web application (an online classifieds server) requires a set of
fairly large global arrays which contain vital information that most all
the page scripts rely upon for information such as the category list,
which fields belong to each category, and so on. Additionally, there are
a large number of function definitions (more than 13,000 lines of code
in all just for these global definitions).
These global arrays and functions never change between requests.
However, the PHP engine destroys and recreates them every time. After
having spent some serious time doing benchmarking (using Apache Bench),
I have found that this code takes at least 7ms to parse per request on
my dual Xeon 2.4ghz server (Zend Accelerator in use*). This seriously
cuts into my server's peak capacity, reducing it by more than half.
My question is: is there a way to define a global set of variables and
functions ONCE per Apache process, allowing each incoming hit to run a
handler function that runs within a persistent namespace? OR, is it
possible to create some form of shared variable and function namespace
that each script can tap?
AFAIK, mod_python, mod_perl, Java, etc. all allow you to create a
persistent, long-running application with hooks/handlers for individual
Apache requests. I'm surprised I haven't found a similar solution for
PHP.
In fact, according to my work in the past few days, if an application
has a large set of global functions and variable definitions, mod_python
FAR exceeds the performance of mod_php, even though Python code runs
significantly slower than PHP code (because in mod_python you can put
all these definitions in a module that is loaded only once per Apache
process).
The most promising prospect I've come across is FastCGI, which for Perl
and other languages, allows you to run a while loop that sits and
receives incoming requests (e.g. "while(FCGI::accept() >= 0) {..}").
However, the PHP/FastCGI modality seems to basically compare to mod_php:
every request still creates and destroys the entire application
(although the PHP interpreter itself does persist).
Essentially I want to go beyond a persistent PHP *interpreter* (mod_php,
PHP/FastCGI) and create a persistent PHP *application*... any
suggestions?
PHP's model is to be completely sandboxed such that every request is
completely separate from every other. Having a persistent interpreter
as you describe would break that rule and break the infinite horizontal
scalability model of PHP.
Of course, there is nothing that prevents you from storing persistent
data somewhere more permanent. If it is just simple read-only data you
have a lot of options. For example, you could put them in a .ini file
that is only loaded on Apache startup and use get_cfg_var() to fetch
them. If you compile PHP with the --with-config-file-scan-dir switch to
configure a configuration scan directory you can just drop your own ini
file in that directory and it will be read on startup. This is just
key=value pairs and not PHP code, of course.
If you need to do something fancier you can stick things in shared
memory. Many of the accelerators give you access to their shared memory
segments. For example, the CVS version of pecl/apc provides apc_store()
and apc_fetch() which lets you store PHP datatypes in shared memory
directly without needing to serialize/unserialize them.
And finally, the big hammer is to write your own PHP extension. This is
a lot easier than people think and for people who are really looking for
performance this is the only way to go. You can write whatever code you
want in your MINIT hook which only gets called on server startup and in
that hook you can create whatever persistent variables you need, pull
stuff from a DB, etc. At the same time you will likely want to pull
some of your heavier business logic identified by your profiling into
the extension as well. This combination of C and PHP is extremely hard
to beat performance-wise regardless of what you compare it to.
-Rasmus
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Persistent PHP web application?
Josh Whiting wrote: > My web application (an online classifieds server) requires a set of > fairly large global arrays which contain vital information that most all > the page scripts rely upon for information such as the category list, > which fields belong to each category, and so on. For this, you should look into: Putting all this stuff in MySQL, but only getting the data you NEED right now at any given time. This will probably be slower, but it will give you a good base-line benchmark, and prepare you for the next step. Moving all that into LDAP. http://php.net/ldap/ However, given your programming philosophy so far, and the fact that you are worried about 7ms and that you specifically requested some kind of shared memory space in PHP, you should Read This: http://us4.php.net/manual/en/ref.sem.php That pretty much is exactly what you asked for. Be forewarned that few PHP scripters have needed this stuff, and it's not anywhere near as hammered on (read: debugged) as the other options above. > Additionally, there are > a large number of function definitions (more than 13,000 lines of code > in all just for these global definitions). You should look into breaking these up into groups of functionality -- I'm willing to bet you could segment these and have many pages that only call in a few functions. > These global arrays and functions never change between requests. > However, the PHP engine destroys and recreates them every time. After > having spent some serious time doing benchmarking (using Apache Bench), > I have found that this code takes at least 7ms to parse per request on > my dual Xeon 2.4ghz server (Zend Accelerator in use*). This seriously > cuts into my server's peak capacity, reducing it by more than half. I'm not sure I'm right, but several things in this paragraph tweak my gut... You're never going to get that 7ms to go to down *TOO* far if you insist on loading all the data and all the functions for your entire site for pages that don't really really need *ALL* of that... > My question is: is there a way to define a global set of variables and > functions ONCE per Apache process, allowing each incoming hit to run a > handler function that runs within a persistent namespace? OR, is it > possible to create some form of shared variable and function namespace > that each script can tap? Variables would be easy: http://us4.php.net/manual/en/ref.sem.php Functions, not so much... Though I suppose if you created all of this as classes, and instantiated a class and dumped a singleton into shared memory, you MIGHT trick PHP into keeping all its class definitions and whatnot in RAM instead of loading from the hard drive... You could also patch Apache and get all your data into your Apache $_SERVER space, or you could (I think) hack your Apache user's shell and get all the data into their shell environment $_ENV. Those would be a bit more hack-y than using shared memory, but they technically fit what you describe... > PHP/FastCGI) and create a persistent PHP *application*... any > suggestions? This part here suggests an entirely different approach... Depending on your application, you could also consider running a loop in PHP which responds to requests on sockets. http://us4.php.net/manual/en/ref.sockets.php You could then define your own "server protocol" -- Kind of like making up your own HTTP/FTP/RPC rules for your own specific application. So if what your application mostly does is load in all this data and respond to requests, you could write a *SINGLE* PHP application which listened on port 12345 (or whatever port you like) and responded with the data requested. Like writing your own web-server, only it's a _-server where you get to fill in the blank with whatever your application does. > * - Please note that I am using the Zend Accelerator (on Redhat > Enterprise with Apache 1.3) to cache the intermediate compiled PHP code. > My benchmarks (7ms+) are after the dramatic speedup provided by the > accelerator. I wouldn't even bother benchmarking this without the > compiler cache, but it is clear that a compiler cache does not prevent > PHP from still having to run the (ableit precompiled) array and function > definition code itself. Actually, Zend and others could be interested in your comparisons of "with" and "without" cache... To Summarize: The solution that most closely approximates what you think you want is "shared memory" features in PHP. The solution that might lead to a much better application would be to segment the data and functions needed into smaller files, and only suck in the ones you REALLY need. An alternative solution that MIGHT fit what you think you want, if you feel up to defining your own XYZ protocol, and if your only goal is to provide the data/functions to respond to requests, is to write a single PHP application that loads the data/functions once, and then sits there listening on a socket to respond to requests. Hope that helps...
